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Preface  and  Acknowledgements 


Founded  in  2003,  the  Acquisition  Research  Program  (ARP)  has  become  an 
institutional,  multi-disciplinary  entity.  In  2006  the  ARP  made  significant  and  sustaining 
progress  toward  realizing  its  goals  to: 

1.  Position  NPS  as  a  recognized  leader  in  defense  acquisition  research. 

2.  Establish  NPS  acquisition  research  as  an  integral  part  of  policy-making  for 
Department  of  Defense  officials. 

3.  Create  a  stream  of  relevant  information  concerning  the  performance  of  DoD 
Acquisition  policies  with  viable  recommendations  for  continuous  process 
improvement. 

4.  Prepare  the  DoD  workforce  to  participate  in  the  continued  evolution  of  the 
defense  acquisition  process. 

5.  Collaborate  with  other  universities,  think  tanks,  industry  and  government  in 
acquisition  research. 

Since  inception,  over  100  reports  and  papers  have  been  published,  thereby  making  a 
significant  contribution  to  the  body  of  literature  on  the  defense  acquisition  process. 

Through  these  research  products,  ARP  sponsors  are  receiving  substantial  help  with  and 
insights  into  the  pressing  business  issues  of  the  day. 

The  synergy  between  faculty  research  and  student  classroom  instruction  has  been 
exceptional  with  many  relevant  and  current  instructional  materials  emerging  from  research 
products,  thus  enhancing  the  student  educational  experience.  Faculty  are  “refreshed”  in 
defense-relevant  subject  matter,  and  students  are  better  prepared  to  enter  the  acquisition 
work  force.  In  recognition  of  these  successes,  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics  (USD  (AT&L))  provided  $1M  in  funding  for  additional 
projects.  This  funding  expands  the  ARP  by  30%  and  is  a  pilot  for  future  increases  in 
research  funding. 

Researcher  opportunities  provided  by  the  Chair  offer  significant  benefits  to 
researchers:  (1)  provision  of  funding  saving  researchers  “marketing"  time;  (2)  ties  with 
sponsor  POCs  thus  assuring  DOD  relevant  research;  (3)  assistance  with  final  formatting, 
editing  and  publishing  thus  relieving  researchers  from  the  “non-intellectual”  aspects  of  their 
research.  Each  of  these  is  a  substantial  benefit  but  the  growing  connectivity  between 
researchers  and  sponsors  is  paying  large  dividends  to  all  concerned.  While  we  at  the  Naval 
Postgraduate  School  like  to  think  of  our  institution  as  the  world’s  leader  in  defense 
acquisition  research,  we  also  recognize  that,  because  of  our  limited  size  and  resources,  we 
are  able  to  study  only  a  few  of  acquisition’s  myriad  of  complex  issues  and  challenges.  We 
know  that  genuine  progress  in  acquisition  research  can  be  achieved  and  sustained  only  to 
the  extent  that  scholars  from  a  broad  range  of  institutions  and  disciplines  are  engaged  to 
participate.  Once  this  “critical  mass”  of  researchers  is  formed,  we  may  anticipate  that 
acquisition  will  become  a  field  of  its  own,  with  perhaps  a  variety  of  acquisition  journals, 
acquisition  conferences,  and  university  courses  in  acquisition  management  and  policy. 
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Such  intellectual  capacity,  we  may  hope,  will  before  long  prevail  against  acquisition’s 
perennial  and  often  pernicious  problems. 

Accordingly,  the  year  2006  was  especially  significant  for  the  NPS  Acquisition 
Research  Program  in  taking  major  strides  toward  expanding  the  program’s  reach  in 
important  ways  to  other  institutions.  The  number  of  research  institutions  participating  as 
collaborators  grew  to  35  with  the  formation  of  a  Virtual  University  Consortium.  Most 
noteworthy  was,  as  mentioned  above,  our  securing  sponsorship  from  USD(AT&L)  to  fund 
research  proposals  selected  from  a  nationwide  call,  or  Broad  Agency  Announcement  (BAA) 
(copy  available  at  www.acquisitionresearch.org).  We’re  truly  excited  at  the  prospects  of 
receiving  innovative  and  cutting  edge  proposals  from  the  top  minds  around  the  country.  We 
trust  that  this  new  sponsorship  will  act  like  good  seeds  sown  in  fertile  soil,  yielding  rich  fruits 
of  profitable  acquisition  research  for  many  years  to  come. 

We  gratefully  acknowledge  the  ongoing  support  and  leadership  of  our  sponsors, 
whose  foresight  and  vision  have  assured  the  continuing  success  of  the  Acquisition 
Research  Program: 

•  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics) 

•  Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition) 

•  Program  Manager  (Infantry  Combat  Equipment) 

•  Program  Executive  Officer  (Integrated  Warfare  Systems) 

•  Program  Executive  Officer  (Littoral  and  Mine  Warfare) 

•  Project  Manager  (Modular  Brigade  Enhancements) 

•  Program  Executive  Officer  (Ships) 

•  Dean  of  Research,  Naval  Postgraduate  School 

We  also  thank  UGS  Corporation  and  the  Naval  Postgraduate  School  Foundation  and 
acknowledge  their  generous  contributions  in  support  of  this  symposium. 


James  B.  Greene,  Jr. 

Rear  Admiral,  US  Navy  (ret) 


Keith  F.  Snider,  PhD 
Associate  Professor 
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The  NPS  “A  Team” 


Rear  Admiral  James  B.  Greene,  Jr.  USN  (Ret.)  —  Acquisition  Chair,  Naval 
Postgraduate  School.  RADM  Greene  develops,  implements  and  oversees  the  Acquisition 
Research  Program  in  the  Graduate  School  of  Business  and  Public  Policy.  He  interfaces  with 
the  DoD,  industry  and  government  leaders  in  acquisition,  supervises  student  MBA  projects 
and  conducts  guest  lectures  and  seminars.  Before  serving  at  NPS,  RADM  Greene  was  an 
independent  consultant  focusing  on  Defense  Industry  business  development  strategy  and 
execution  (for  both  the  public  and  private  sectors),  minimizing  lifecycle  costs  through 
technology  applications,  alternative  financing  arrangements  for  capital-asset  procurement, 
and  “red-teaming”  corporate  proposals  for  major  government  procurements. 

RADM  Greene  served  as  the  Assistant  Deputy  Chief  of  Naval  Operations  (Logistics) 
in  the  Pentagon  from  1991-1995.  As  Assistant  Deputy,  he  provided  oversight,  direction  and 
budget  development  for  worldwide  US  Navy  logistics  operations.  He  facilitated  depot 
maintenance,  supply-chain  management,  base/station  management,  environmental 
programs  and  logistic  advice  and  support  to  the  Chief  of  Naval  Operations.  Some  of  his 
focuses  during  this  time  were  leading  Navy-wide  efforts  to  digitize  all  technical  data  (and, 
therefore,  reduce  cycle  time)  and  to  develop  and  implement  strategy  for  procurement  of 
eleven  Sealift  ships  for  the  rapid  deployment  forces.  He  also  served  as  the  Senior  Military 
Assistant  to  the  Under  Secretary  of  Defense  (Acquisition)  from  1987-1990  where  he  advised 
and  counseled  the  Under  Secretary  in  directing  the  DoD  procurement  process. 

From  1984-1987,  RADM  Greene  was  the  Project  Manager  for  the  Aegis  project.  This 
was  the  DoD’s  largest  acquisition  project  with  an  annual  budget  in  excess  of  $5  Billion/year. 
The  project  provided  oversight  and  management  of  research,  development,  design, 
production,  fleet  introduction  and  full  lifecycle  support  of  the  entire  fleet  of  Aegis  cruisers, 
destroyers  and  weapons  systems  through  more  than  2500  industry  contracts.  From  1980- 
1984,  RADM  Greene  served  as  Director,  Committee  Liaison,  Office  of  Legislative  Affairs 
followed  by  a  tour  as  the  Executive  Assistant,  to  the  Assistant  Secretary  of  the  Navy 
(Shipbuilding  and  Logistics).  From  1964-1980,  RADM  Greene  served  as  a  Surface  Warfare 
Officer  in  various  duties,  culminating  in  Command-at-Sea.  His  assignments  included 
numerous  wartime  deployments  to  Vietnam  as  well  as  the  Indian  Ocean  and  the  Persian 
Gulf. 


RADM  Greene  received  a  BS  in  Electrical  Engineering  from  Brown  University  in 
1964;  he  earned  a  MS  in  Electrical  Engineering  and  a  MS  in  Business  Administration  from 
the  Naval  Postgraduate  School  in  1973. 

Keith  F.  Snider  —  Associate  Professor  of  Public  Administration  and  Management  in 
the  Graduate  School  of  Business  &  Public  Policy  at  the  Naval  Postgraduate  School  in 
Monterey,  California,  where  he  teaches  courses  related  to  defense  acquisition  management. 
He  also  serves  as  Principal  Investigator  for  the  NPS  Acquisition  Research  Program  and  as 
Academic  Associate  for  resident  NPS  acquisition  curricula. 

Professor  Snider  has  a  PhD  in  Public  Administration  and  Public  Affairs  from  Virginia 
Polytechnic  Institute  and  State  University,  a  Master  of  Science  degree  in  Operations 
Research  from  the  Naval  Postgraduate  School,  and  a  Bachelor  of  Science  degree  from  the 
United  States  Military  Academy  at  West  Point.  He  served  as  a  field  artillery  officer  in  the  US 
Army  for  twenty  years,  retiring  at  the  rank  of  Lieutenant  Colonel.  He  is  a  former  member  of 
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the  Army  Acquisition  Corps  and  a  graduate  of  the  Program  Manager’s  Course  at  the 
Defense  Systems  Management  College. 

Professor  Snider’s  recent  publications  appear  in  American  Review  of  Public 
Administration,  Administration  and  Society,  Administrative  Theory  &  Praxis,  Journal  of  Public 
Procurement,  Acquisition  Review  Quarterly,  and  Project  Management  Journal. 

Karey  L.  Shaffer  —  Program  Manager  for  the  Acquisition  Research  Program  at  the 
Graduate  School  of  Business  and  Public  Policy,  Naval  Postgraduate  School.  As  PM,  Shaffer 
is  responsible  for  operations  and  publications  in  conjunction  with  the  Acquisition  Chair  and 
the  Principal  Investigator.  She  has  also  catalyzed,  organized  and  managed  the  Acquisition 
Research  Symposiums  hosted  by  NPS. 

Shaffer  has  also  served  as  an  independent  Project  Manager  and  Marketing 
Consultant  on  various  projects.  Her  experiences  as  such  were  focused  on  creating 
marketing  materials,  initiating  web  development,  assembling  technical  teams,  managing 
project  lifecycles,  processes  and  cost-savings  strategies. 

From  2001-2002,  Shaffer  contracted  to  work  as  the  Executive  Assistant  to  the  Vice 
President  for  Leadership  and  Development  Human  Resources  for  Metris  Companies  in 
Minneapolis.  In  this  capacity,  she  introduced  project  lifecycle  and  process  improvements  to 
increase  efficiency.  Likewise,  as  a  Resource  Specialist  contractor  at  Watson  Wyatt 
Worldwide  in  Minneapolis,  she  developed  and  implemented  template  plans  to  address 
continuity  and  functionality  in  corporate  documents;  in  this  same  position,  she  introduced 
process  improvements  to  increase  efficiency  in  presentation  and  proposal  production  in 
order  to  reduce  the  instances  of  corruption  and  loss  of  vital  technical  information. 

Shaffer  has  also  served  as  the  Project  Manager  for  Imagicast,  Inc.  and  as  the 
Operations  Manager  for  the  Montana  World  Trade  Center.  At  Imagicast,  she  was  asked  to 
take  over  the  project  management  of  four  failing  pilots  for  Levi  Strauss  in  the  San  Francisco 
office.  Within  four  months,  the  pilots  were  released;  the  project  lifecycle  was  shortened;  and 
the  production  process  was  refined.  In  this  latter  capacity  at  the  MWTC,  Shaffer  developed 
operating  procedures,  policies  and  processes  in  compliance  with  state  and  federal  grant 
law.  Concurrently,  she  managed  $1.25  million  in  federal  appropriations,  developed 
budgeting  systems  and  secured  a  $400,000  federal  technology  grant.  As  the  Operations 
Manager,  she  also  designed  MWTC’s  Conference  site,  managed  various  marketing 
conferences,  and  taught  student  practicum  programs  and  seminars. 

Shaffer  has  her  BA  in  Business  Administration  (focus  on  International  Business, 
Marketing  and  Management)  from  the  University  of  Montana.  She  is  currently  earning  her 
MBA  from  San  Francisco  State  University. 

A  special  thanks  to  our  editor  Jeri  Larsen  for  all  that  she  has  done  to  make  this 
publication  a  success,  to  David  Wood  and  Carl  Matsen  for  production,  to  Ian  White  for 
graphic  support,  to  Lindsay  D’Penha  for  CD  programming,  to  Jordy  Boom  for  conference 
website  development.  We  would  like  acknowledge  Arlene  Pulido,  Jennifer  Watson,  Bon 
Troung,  Toan  Tran  and  Jason  Munoz  of  the  staff  at  the  Graduate  School  of  Business  & 
Public  Policy  for  all  the  administrative  support  on  the  backend  to  make  the  Symposium  a 
success.  Our  program  success  is  directly  related  to  the  combined  efforts  of  many. 


w»  kbwtu„ 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  iv  - 


ACQUISITION  RESEARCH: 


CREATING  SYNERGY  FOR  INFORMED  CHANGE 


5th  Annual  Acquisition  Research  Symposium 


May  14-15,  2008 


Announcement  and  Call  for  Proposals 


The  Graduate  School  of  Business  &  Public  Policy  at  the  Naval  Postgraduate  School 
announces  the  5th  Annual  Acquisition  Research  Symposium  to  be  held  May  14-15,  2008 
in  Monterey,  California. 

This  symposium  serves  as  a  forum  for  the  presentation  of  acquisition  research  and 
the  exchange  of  ideas  among  scholars  and  practitioners  of  public-sector  acquisition.  We 
seek  a  diverse  audience  of  influential  attendees  from  academe,  government,  and  industry 
who  are  well  placed  to  shape  and  promote  future  research  in  acquisition. 

The  Symposium  Program  Committee  solicits  proposals  for  panels  and/or  papers 
from  academicians,  practitioners,  students  and  others  with  interests  in  the  study  of 
acquisition.  The  following  list  of  topics  is  provided  to  indicate  the  range  of  potential  research 
areas  of  interest  for  this  symposium:  acquisition  and  procurement  policy,  supply  chain 
management,  public  budgeting  and  finance,  cost  management,  project  management, 
logistics  management,  engineering  management,  outsourcing,  performance 
measurement,  and  organization  studies. 

Proposals  must  be  submitted  by  November  9,  2007.  The  Program  Committee  will 
make  notifications  of  accepted  proposals  by  December  7,  2007.  Final  papers  must  be 
submitted  by  April  4,  2008  to  be  included  in  the  Symposium  Proceedings. 

Proposals  for  papers  should  include  an  abstract  along  with  identification,  affiliation, 
and  contact  information  for  the  author(s).  Proposals  for  papers  plan  for  a  20  minute 
presentation.  Proposals  for  panels  (plan  for  90  minute  duration)  should  include  the  same 
information  as  above  as  well  as  a  description  of  the  panel  subject  and  format,  along  with 
participants’  names,  qualifications  and  the  specific  contributions  each  participant  will  make 
to  the  panel. 

Submit  paper  and  panel  proposals  to  www.researchsymposium.org  . 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  v  - 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


Twiskmw  rw  Kir  si"™ 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  vi  - 


NPS  BAA-07-002 


BROAD  AGENCY  ANNOUNCEMENT 


Acquisition  Research  Program 


Naval  Postgraduate  School 


Open  until  4:00  pm  PDST  1  June  2007 


Primary  objective  is  to  attract  outstanding  researchers  and  scholars  to 
investigate  topics  of  interest  to  the  defense  acquisition  community.  The 

program  solicits  innovative  proposals  for  defense  acquisition  management  and 
policy  research  to  be  conducted  during  fiscal  year  (FY)  2008  (1  Oct  07  -30  Sep  08). 

Defense  acquisition  management  and  policy  research  refers  to 
investigations  in  all  disciplines,  fields,  and  domains  that  (1)  are  involved  in  the 
acquisition  of  products  and/or  services  for  national  defense,  or  (2)  could 
potentially  be  brought  to  bear  to  improve  defense  acquisition.  It  includes  but  is 
not  limited  to  economics,  finance,  financial  management,  information  systems, 
organization  theory,  operations  management,  human  resources  management,  and 
marketing,  as  well  as  the  “traditional”  acquisition  areas  such  as  contracting, 
program/project  management,  logistics,  and  systems  engineering  management. 

This  program  is  targeted  in  particular  to  U.S.  universities  (including  U.S. 
government  schools  of  higher  education)  or  other  research  institutions 
outside  the  Department  of  Defense. 

The  Government  anticipates  making  multiple  awards  up  to  $100,000  each  for 
a  basic  research  period  of  twelve  months.  NPS  plans  to  complete  proposal 
evaluations  and  notify  awardees  in  early  August  2007. 


Full  Text  for  NPS  BAA-07-002 


at 


http://www.nps.edu/Research/WorkinqWithNPS.html 
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Keynote  Speaker 


Thursday, 

May  17,  2007 

Keynote  Speaker 

8:00  a.m.  - 

Keynote  Speaker 

9:15  a.m. 

Mr.  Shay  D.  Assad  -  Director,  Defense  Procurement  and  Acquisition 

Policy,  Office  of  the  Under  Secretary  of  Defense  (Acquisition,  Technology 
and  Logistics) 

Keynote  Speaker:  Mr.  Shay  Assad  assumed  his  position  as 
Director  of  Defense  Procurement  and  Acquisition  Policy  (DPAP) 
on  April  3,  2006.  As  the  Director  of  DPAP,  he  is  responsible  for  all 
acquisition  and  procurement  policy  matters  in  the  Department  of 
Defense  (DoD).  He  serves  as  the  principal  advisor  to  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics, 
the  Deputy  Under  Secretary  for  Acquisition  and  Technology,  and 
the  Defense  Acquisition  Board  on  acquisition/procurement 
strategies  for  all  major  weapon  systems  programs,  major 
automated  information  systems  programs,  and  services 
acquisitions.  He  is  responsible  for  procurement/sourcing 
functional  business  process  requirements  in  the  Department’s 
Business  Enterprise  Architecture  (BEA)  and  Enterprise  Transition  Plan  (ETP).  Mr.  Assad  is 
DoD’s  advisor  for  competition,  source  selection,  multiyear  contracting,  warranties,  leasing  and 
all  international  contracting  matters. 

Before  assuming  this  position,  Mr.  Assad  was  the  Assistant  Deputy  Commandant,  Installations 
and  Logistics  (Contracts),  Headquarters,  Marine  Corps,  Washington,  D.C.  He  had  held  the 
position  as  the  Marine  Corps’  senior  civilian  contracting  official  since  June  2004. 

Upon  graduating  with  distinction  from  the  U.S.  Naval  Academy  in  1972,  he  served  two  tours  of 
duty  aboard  U.S.  Navy  destroyers  and  won  recognition  as  Outstanding  Junior  Officer,  Fifth 
Naval  District.  He  then  served  as  a  Naval  Procurement  Officer  at  the  Naval  Sea  Systems 
Command,  where  he  was  responsible  for  the  negotiation  and  administration  of  the  Aegis 
Weapons  Systems  engineering  and  production  contracts. 

In  1978,  Mr.  Assad  began  working  for  the  Raytheon  Company.  Between  1978  and  1994  he 
served  in  several  increasingly  responsible  contract  management  positions  in  Raytheon's  largest 
Electronics  and  Missile  divisions  gaining  extensive  experience  in  defense,  commercial  and 
international  contracting.  He  was  intimately  involved  in  numerous  major  programs  such  as 
PATRIOT,  HAWK,  AMRAAM,  PAVE  PAWS,  BMEWS,  Standard  Missile,  Aegis  ER,  Sparrow  and 
Ballistic  Missile  Defense.  In  1994,  he  was  promoted  to  Vice  President  -  Director  of  Contracts  for 
Raytheon,  and  was  subsequently  promoted  to  Senior  Vice  President,  Contracts  in  1997.  As 
such,  he  was  responsible  for  the  contract  negotiation  and  administration  activities  ($20  Billion)  in 
all  of  Raytheon's  businesses  -  both  government  and  commercial. 
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In  addition  to  his  contracting  duties,  Mr.  Assad  was  given  numerous  program  and  business 
management  special  assignments  by  Raytheon's  Executive  Office.  These  assignments 
spanned  participation  in  all  three  of  Raytheon's  major  operating  businesses  -  Government, 
Aviation,  and  Engineering  and  Construction. 

In  1998  he  was  promoted  to  Executive  Vice  President  and  served  as  the  Chief  Operating  Officer 
and  subsequently,  as  the  Chairman  and  Chief  Executive  Officer  of  Raytheon's  Engineering  and 
Construction  (RE&C)  business.  RE&C  was  a  $2.7  billion  international  company  with  more  than 
15,000  professional  and  craft  employees,  24  product  lines  and  1 1  major  offices  worldwide. 

After  his  retirement  from  the  Raytheon  Company,  in  2001,  Mr.  Assad  established  a  small 
business  primarily  providing  consulting  and  retail  services. 
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Panel  11  -  Strategic  Financing  of  DoD  Resources  and  Budget 
Policies 


Thursday, 
May  17  2007 


9:15  a.m.-  Chair: 
11:00. m. 


Donald  Summers,  Lecturer,  Naval  Postgraduate  School 

Discussants: 

William  Eggers,  Global  Research  Director,  Deloitte  Research  Public  Sector 

E.  Sanderson  (Sandy)  Hoe,  Partner  at  McKenna,  Long  &  Aldridge,  LLP 

Jim  Hearn,  Director  for  Federal  Programs  and  Budget  Process,  Senate 
Budget  Committee 

Papers: 

The  Folly  of  Consequence-free  Budget  Scoring 

Gerald  S.  Koenig,  Hannon  Armstrong  Advisory  Services 

The  Budget  Scoring  Alternatives  Financing  Methods  for  Defense 
Requirements 

LT  Leonard  Leos,  USN,  LT  Paul  Rouleau,  USN,  and  LT  Mark 
Wadsworth,  USN,  Naval  Postgraduate  School 


Chair:  Donald  E.  Summers,  LtCol,  USMC  (Ret.),  CMA,  CFM  is  Lecturer  in  Financial  Management, 
Graduate  School  of  Business  &  Public  Policy,  Naval  Postgraduate  School.  He  received  an  MS  from  the 
Naval  Postgraduate  School.  Before  retiring  from  the  USMC,  he  was  Program  Budget  Coordinator  for  the 
Chief  Financial  Officer,  Headquarters  Marine  Corps.  He  has  been  employed  as  a  senior  financial  analyst 
for  Household  Credit  Services  and  as  the  executive  assistant  to  the  Chairman/CEO  (Mr.  Bill  Agee)  of 
Morrison  Knudson  Corporation.  His  interests  are  financial  &  managerial  accounting,  federal  budgeting 
and  armed  forces  comptrollership. 

Discussant:  William  Eggers,  Global  Research  Director,  Deloitte  Research  Public  Sector,  is  a  Senior 
Fellow  at  the  New  York-based  Manhattan  Institute  for  Policy  Research.  He  examines  how  technology  can 
be  used  to  reinvent  government  structures  for  greater  efficiency.  In  addition  he  is  the  Global  Director  for 
Deloitte  Research-Public  Sector,  where  he  is  responsible  for  research  and  thought  leadership  for 
Deloitte’s  public-sector  practice. 

He  is  the  author  of  two  new  books  on  transforming  government:  Government  2.0:  Using  Technology  to 
Improve  Education,  Cut  Red  Tape,  Reduce  Gridlock,  and  Enhance  Democracy  (Rowman  and  Littlefield, 
2005)  and  Governing  by  Network:  The  New  Shape  of  the  Public  Sector  (Brookings,  2004). 

Eggers  is  a  former  appointee  to  the  US  Office  of  Management  and  Budget’s  Performance  Measurement 
Advisory  Commission  and  the  former  Project  Director  for  the  Texas  Performance  Review/  e-Texas 
initiative.  The  two  performance  reviews  Eggers  was  involved  in  identified  over  $2.5  billion  worth  of 
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savings  and  non-tax  revenues  for  the  state.  Over  60  percent  of  the  recommendations  in  the  reviews  were 
enacted  into  law.  Mr.  Eggers  also  served  as  a  Commissioner  for  the  Texas  Incentive  and  Productivity 
Commission  and  a  designee  on  the  Texas  Council  on  Competitive  Government. 

In  addition,  Eggers  was  the  Chair  of  the  Government  Reform  Policy  Committee  for  then  Governor  George 
W.  Bush  during  his  first  presidential  campaign.  In  this  capacity  he  coordinated  research  for  the  campaign 
in  government  management  and  reform  issues. 

Mr.  Eggers  is  the  former  Director  of  Government  Reform  at  the  Reason  Public  Policy  Institute,  a  Los 
Angeles-based  think  tank.  A  nationally  recognized  expert  on  government  reform,  Eggers  is  the  1996 
winner  of  the  prestigious  Roe  Award  for  leadership  and  innovation  in  public  policy  research  and  the  2002 
APEX  award  for  excellence  in  business  journalism.  Mr.  Eggers  is  also  the  co-author  of  Revolution  at  the 
Roots:  Making  our  Government  Smaller,  Better,  and  Closer  to  Home  (The  Free  Press).  The  book  was 
named  the  winner  of  the  1 996  Sir  Anthony  Fisher  International  Memorial  Award  for  the  book  "making  the 
greatest  contribution  to  the  understanding  of  the  free  economy  during  the  past  two  years." 

Prior  to  joining  the  Reason  Foundation,  Mr.  Eggers  assisted  reformers  in  Eastern  Europe  and  the  former 
Soviet  Union  with  the  transition  from  socialist  to  free-market  economies  as  a  policy  analyst  at  The 
Heritage  Foundation  in  Washington,  D.C. 

Mr.  Eggers  has  advised  dozens  of  cities,  states,  and  foreign  countries  and  trained  hundreds  of  public 
officials  on  restructuring  government. 

Mr.  Eggers  graduated  magna  cum  laude  from  the  University  of  California  at  San  Diego. 

Discussant:  E.  Sanderson  (Sandy)  Hoe,  Partner  at  McKenna,  Long  &  Aldridge,  LLP,  work 
encompassed  counseling  and  litigation  on  behalf  of  clients  on  a  broad  range  of  government  contracts 
laws  and  regulations.  His  expertise  extends  from  issues  in  contract  formation,  including  bid  protests,  the 
negotiation  of  subcontracts  and  the  structuring  of  complex  private  financing,  to  the  preparation  of  complex 
claims,  and  to  resolution  of  post-award  contract  disputes  through  litigation  or  alternate  dispute  resolution 
methods.  His  clients  include  major  companies  in  the  defense,  telecommunications,  information 
technology,  financial,  and  health  care  industries. 

Sandy  has  counseled  clients  on  government  socio-economic  programs,  the  flowdown  of  requirements  to 
government  subcontractors,  the  application  of  international  agreements  such  as  the  Buy  American  Act 
and  Trade  Agreements  Act  to  government  contractors  and  a  host  of  other  issues.  He  has  litigated  multi¬ 
million  dollar  contract  disputes  involving  government  allegations  of  contractor  defective  pricing  and  failure 
to  perform.  His  litigation  experience  includes  challenges  to  government  refusal  fully  to  reimburse 
contractors  terminated  for  convenience.  He  has  settled  major  prime  -  subcontractor  disputes  through 
mediation.  He  served  on  the  ABA  committee  that  prepared  model  flow-down  terms  and  conditions  for 
subcontracts  under  government  prime  contracts  following  adoption  of  the  Federal  Acquisition  Regulation 
in  1 984.  He  also  is  expert  on  efforts  by  the  government  to  outsource  work  to  the  private  sector  and  co¬ 
chairs  the  ABA's  Public  Contract  Law  Section,  Privatization  and  Competitive  Sourcing  Committee. 

In  the  area  of  finance,  and  to  help  clients  take  advantage  of  government  initiatives  to  privatize  certain  of 
its  activities,  Sandy  successfully  has  counseled  clients  in  the  creation  of  third-party  financed  government 
contracts,  including  long-term  leases,  and  related  ancillary  agreements  for  such  items  as  ships,  aircraft, 
military  housing,  energy  facilities  and  information  technology.  Clients  in  this  field  have  included  major 
contractors,  banks,  investment  bankers  and  insurance  companies. 

Sandy  is  the  author  of  "Law  for  Engineers"  (George  Washington  University,  1 980)  and  "Flow-Down 
Clauses  in  Subcontracts,"  Briefing  Papers  (Federal  Publications,  Inc.).  He  was  a  professorial  lecturer  in 
Engineering  Law  at  George  Washington  University  (1980-1984)  and  has  lectured  for  Federal 
Publications,  Inc.,  the  National  Contract  Management  Association,  the  American  Bar  Association,  and 
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industry  groups.  He  served  for  six  years  on  the  Steering  Committee  of  the  Section  on  Government 
Contracts  and  Litigation  of  the  District  of  Columbia  Bar,  including  three  years  as  co-chair. 

For  several  years,  Sandy  also  practiced  telecommunications  regulatory  law,  appearing  before  numerous 
state  public  utility  commissions  in  hearings  to  open  the  local  exchange  markets  for  new  entrants  under 
the  Telecommunications  Act  of  1996. 

Discussant:  Jim  Hearn,  Director  for  Federal  Programs  and  Budget  Process,  Senate  Budget 
Committee 
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The  Folly  of  Consequence-free  Budget  Scoring 


Presenter:  Gerald  S.  Koenig  is  Managing  Director  of  Hannon  Armstrong  Advisory  Services,  a  firm  that 
advises  government  and  corporate  clients  on  the  economics  and  financing  of  critical  infrastructure  needs. 
He  has  over  twenty  years  of  experience  in  senior  executive  and  legal  positions  and  served  both  on  the 
White  House  staff  under  President  Ronald  Reagan  and  in  the  Office  of  Secretary  of  Defense  under 
Richard  Cheney.  Mr.  Koenig  received  a  BS  in  engineering  from  West  Point,  an  MA  in  International 
Relations  from  USC,  and  a  JD  in  International  Business  Transactions  from  the  George  Mason  University 
School  of  Law;  he  is  a  member  of  the  District  of  Columbia  Bar. 

HA  Advisory  Services  advises  government  and  corporate  clients  on  how  to  use  multiyear  Federal  funding 
to  acquire  critical  infrastructure  needs.  HA  Advisory  Services  is  an  affiliate  of  Hannon  Armstrong  Capital, 
which  for  over  25  years  has  provided  financing  for  government  infrastructure  projects. 


Things  and  actions  are  what  they  are,  and  the 
consequences  of  them  will  be  what  they  will  be; 
why,  then,  should  we  desire  to  be  deceived? 


Bishop  Joseph  Butler 
1692-1752 


Overview 

Current  Congressional  Budget  Office  (CBO)  budget  scoring  rules  cheat  taxpayers  and 
warfighters  by  ignoring  the  high  cost  of  not  acquiring  cost-effective  upgrades  to  critical  combat 
weapons.  Treating  paid-over-time  procurements  as  if  they  are  paid-up-front  budget  outlays 
necessarily  perpetuates  waste  and  inefficiency  where  we  can  least  afford  it:  on  the  modern 
battlefield.  As  a  result,  the  current  acquisition  process  for  such  upgrades  involves  a  simplistic, 
two-step  process.  First,  determine  if  paying  the  entire  cost  up-front  of  an  upgrade  is  less 
expensive  than  the  net  present  value  of  paying  for  the  upgrade  over  time.  Once  paying  up-front 
is  “discovered”  to  be  the  cheaper  option  (as  nearly  always  occurs),  the  next  step  is  to  abandon 
the  upgrade  as  soon  as  it  fails  to  compete  successfully  for  scarce  procurement  budget  dollars. 
An  extremely  conscientious  program  official  may  repeat  this  process  for  a  number  of  budget 
cycles.  But  in  the  end,  the  outcome  is  predictable.  The  game  is  just  rigged  that  way. 

The  problem  is  that  these  policies  have  real  consequences  that  squander  taxpayer 
dollars  while  degrading  battlefield  performance.  Many  of  America’s  major  combat  weapons 
systems  have  engines  that  are  so  old  and  obsolete  that  modern  upgrades  would  easily  pay  for 
themselves  in  fuel  and  maintenance  savings  while  dramatically  increasing  combat  range  and 
battlefield  reliability.  The  private  sector  is  willing  to  give  the  DoD  such  upgrades  at  no  upfront 
cost  in  exchange  for  annual  “mortgage-like”  payments  that  are  never  greater  than  verified 
savings  in  fuel  and  maintenance.  How  can  we  know  this?  Because  the  DoD  has  routinely  used 
such  “paid-from-savings”  contracts  for  over  a  decade  to  upgrade  infrastructure  on  military 
bases.  Similar  contracts  are  widely  used  by  the  DoD  to  acquire  vast  amounts  of  information 
technology  and  telecommunication  assets. 

But  when  legislation  was  introduced  in  Congress  to  extend  paid-from-savings  contracts 
to  combat  platforms,  the  CBO  “scored”  the  legislation  so  high  that  it  effectively  killed  the 
legislative  effort.  The  CBO  insists  that  paid-over-time  acquisitions  should  “score”  for  budget 
purposes  in  the  same  manner  as  if  an  outright  purchase  was  made — even  if  the  payments-over- 
time  are  limited  by  law  to  the  verified  savings  produced  by  the  acquisition.  By  counting  the 
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payments  and  ignoring  the  savings  as  “speculative,”  the  CBO  ensures  wasteful  outcomes;  the 
more  an  upgrade  saves,  the  more  the  CBO  will  assert  it  costs. 

The  CBO  claims  that  accounting  for  acquisitions  in  any  other  manner  is  inconsistent  with 
the  Budget  Act  of  1 967.  The  fact  is  that  that  Act  and  all  Federal  budget  laws  are  silent  on  this 
issue.  The  CBO’s  position  is  really  a  hunch — no  more  than  one  interpretation  of  general 
principles.  The  CBO’s  counterpart  in  the  Executive  Branch,  the  Office  of  Management  and 
Budget  (OMB),  reaches  the  opposite  conclusion. 

The  CBO  views  its  scoring  policy  in  a  consequence-free  vacuum  and  sees  its  role  as  a 
dispassionate  arbiter  of  how  to  apply  general  budget  principles  to  specific  legislation.  Actual 
outcomes  are  seen  as  irrelevant — or,  at  the  very  least,  as  highly  speculative  and,  therefore, 
properly  dismissed. 

Supporting  this  dysfunctional  acquisition  system  is  an  almost  smug  attitude  among  many 
defense  acquisition  policy  apparatchiks.  Unfortunately,  military  and  civilian  officials  who  should 
be  fierce  advocates  for  warfighters  on  such  policy  issues  are  missing  in  action  on  this  issue. 
Since  the  Boeing  tanker  lease  scandal,  anyone  who  challenges  the  orthodoxy  of  traditional 
defense  procurement  is  considered,  at  best,  politically  tone-deaf. 

Government  accounting  procedures  should  serve  the  mission  of  the  government — not 
vice-versa.  If  the  National  defense  mission  of  the  Federal  government  is  important  enough  to 
siphon  off  wealth  from  citizens  in  the  form  of  taxes,  the  first  rule  should  be  that  every  tax  dollar 
must  buy  the  most  combat  power  possible.  Budget  rules  that  frustrate  this  purpose  need  to  be 
changed. 

Why  Inefficiency  Matters  on  Combat  Platforms 

The  President’s  2006  State  of  the  Union  address  should  resonate  in  many  quarters, 
including  in  the  President’s  own  Executive  Branch.  Nowhere  is  the  “addiction  to  oil”  that  the 
President  cited  more  serious  than  in  the  Federal  government,  which  enjoys  the  distinction  of 
being  the  largest  single  consumer  of  energy  in  the  world.  Within  the  Federal  government,  the 
Department  of  Defense  leads  all  agencies  in  consumption  of  oil.  This  is  not  surprising, 
considering  the  vast  arsenal  of  tanks,  ships,  aircraft  and  bases  that  the  DoD  uses  in  its  critical 
warfighting  operations. 

What  is  surprising,  however,  is  how  brave  Americans  are  sent  into  battle  with  obsolete 
oil-consuming  systems  that  would  be  cheaper  to  replace  with  state-of-the-art  upgrades.  Some 
of  our  most  famous  aircraft  and  other  weapons  systems  are  long  overdue  for  new  engines  since 
their  forty-year-old  engines  are  underpowered  and  undependable — and  waste  billions  of 
taxpayer  dollars  on  fuel. 

Breaking  any  addiction  requires  that  one  first  admit  there  is  a  problem.  As  taxpayers,  we 
have  a  problem  with  Federal  accounting  rules  that  are  rigged  to  waste  our  tax  dollars.  As  for 
warfighters,  their  problem  is  that  these  same  policies  send  them  into  battle  with  second-rate 
equipment.  But  all  of  us  have  a  common  problem:  Federal  accountants  clearly  could  not  care 
less  about  these  outcomes;  their  attitude  is  simple:  It’s  not  my  department. 

Since  the  American  Revolution,  how  the  government  buys  military  goods  and  services 
been  a  source  of  constant  concern — and  with  good  reason.  Failing  to  get  the  most  for  each 
taxpayer  dollars  is  always  bad,  but  it  is  hard  to  imagine  a  greater  abomination  than  when 
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American  warfighters  are  sent  into  battle  with  second-rate  equipment  when  first-rate  equipment 
could  bought  at  less  cost.  Historically,  war  profiteers  and  corrupt  bureaucrats  are  the  usual 
suspects,  but  here  the  culprits  are  myopic  budget  officials. 

Most  Americans  would  be  surprised  to  learn  that  some  of  our  best-known  weapons  are 
grossly  underpowered  by  forty-year-old  engines  that  cheat  warfighters  of  combat  range  and 
power  while  cheating  taxpayers  by  guzzling  gas  and  requiring  ever-increasing  amounts  of 
maintenance.  Reengining  with  state-of-the-art  engines  could  give  warfighters  the  best  “bang  for 
the  buck”  while  taxpayers  would  harvest  a  windfall  in  savings.  The  savings  are  so  great  and  so 
certain  that  the  private  sector  has  offered  the  DoD  a  compelling  offer:  Let  us  give  you  the  new 
engines,  and  you  can  pay  us  back  from  the  savings — but  only  if  and  when  those  savings 
materialize. 

The  DoD  routinely  accepts  such  offers  when  the  stakes  are  low,  such  as  upgrading 
energy  systems  on  military  bases.  Congress  created  Energy  Savings  Performance  Contracts  in 
the  1990s  for  exactly  this  purpose.  Since  then,  billions  of  taxpayer  dollars  have  been  saved 
using  these  “paid-from-savings-over-time”  contracts.  The  advantage  is  not  just  a  matter  of 
making  defense  contractors  guarantee  savings.  The  real  advantage  is  that  these  contracts 
break  the  cycle  in  which  aging  engines  rarely  compete  successfully  for  full,  up-front  funding  in 
the  Federal  budget  process. 

That  cycle  is,  after  all,  how  these  assets  got  to  be  old  and  obsolete  in  the  first  place. 
Nevertheless,  many  OMB  and  CBO  budget  officials  fail  to  see  this  is  a  problem,  despite  it  being 
documented  in  numerous  DoD  and  Air  Force  studies  and  reports  to  Congress.  The  accountants 
remain  fixed  in  their  belief  that  if  replacing  obsolete  engines  was  important  enough,  the  DoD’s 
limited  capital  budget  would  be  allocated  to  that  purpose.  These  same  accountants  are  not 
bothered  that  the  other  major  part  of  the  defense  spending,  the  “operations  and  maintenance” 
budget,  is  being  drained  by  these  gas-guzzling  maintenance  hogs  year-in  and  year-out — even 
when  offered  a  paid-from-savings  solution  that  pays  for  the  new  engines  out  of  savings  from 
existing  operations  and  maintenance  budget  levels.  Tough  luck,  soldier,  is  their  bottom  line. 

Of  course,  buying  anything  over  time  is  more  expensive  than  paying  cash  up-front.  But 
as  long  as  there  is  a  National  Debt,  even  capital  appropriations  from  Congress  cost  the 
taxpayers  interest.  Thus,  the  issue  is  really  how  much  more  interest  does  an  Energy  Savings 
Performance  contract  cost  the  taxpayers,  and  what  do  the  taxpayers  get  for  that  extra  cost? 
Moreover,  the  real  world  choice  is  not  just  between  buying  outright  or  buying  over  time.  The 
third — and  most  often  selected — choice  is  simply  to  do  nothing. 

Unfortunately,  doing  nothing  can  be  the  worst  choice  of  all.  This  is  exactly  what  is 
occurring  in  a  surprising  number  of  combat  fleets,  from  Abrams  tanks  to  B-52  Bombers.  To  be 
clear,  a  legacy  tank,  aircraft  or  ship  itself  may  still  have  decades  of  useful  life.  After  all,  it’s  hard 
to  wear  out  a  tank.  And  until  some  new  enemy  advances  Panzer-technology,  the  Abrams  tank 
is  unlikely  to  meet  its  match  on  a  battlefield  anytime  soon.  The  same  goes  for  B-52  and  the 
Joint  STARS  aircraft  fleets — and  any  number  of  other  legacy  fleets. 

But  technology  advances  at  different  rates  in  different  areas.  So,  while  these  ships, 
tanks  and  aircraft  may  still  be  perfectly  suited  for  battle,  their  engines  often  are  not.  Engine 
technology  has  advanced  so  profoundly  in  the  last  few  decades  that  state-of-the-art  engines  can 
quickly  pay  for  themselves  in  fuel  and  maintenance  savings — and  do  this  while  providing  greater 
power,  combat  range  and  battlefield  reliability.  A  recent  Air  Force  study  estimated  that 
reengining  the  B-52  fleet  alone  would  pay  for  itself  in  less  than  half  of  the  remaining  life  of  the 
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airframe,  extend  combat  range  by  30%  and  save  so  much  fuel  that  even  in  peacetime,  it  would 
be  equivalent  to  taking  over  144,000  cars  off  the  road  each  year. 

None  of  this  is  disputed  by  the  DoD,  Congress  or  the  White  House.  So,  why  are  we  still 
sending  brave  Americans  into  battle  with  obsolete  equipment  that  costs  taxpayers  more  than 
state-of-the-art  alternatives?  The  reason  is  simple:  Federal  accountants  are  blocking  combat 
upgrades  that  save  money  and  lives  because  their  accounting  rules  are  based  on  bad  logic. 

Just  as  with  energy  system  upgrades  on  military  bases,  there  are  generally  three 
choices  with  obsolete  engines.  First,  you  can  buy  the  upgrade  outright,  assuming  that 
reengining  B-52s,  Joint  STARS  or  Abrams  tanks  can  compete  for  scarce  capital  appropriations. 
Second,  you  can  buy  the  upgrade  overtime,  matching  payments  with  the  savings  produced  by 
the  upgrade,  thereby  spending  no  more  in  any  given  year  than  would  have  otherwise  been 
spent.  Third,  you  can  do  nothing. 

Again,  no  one  disputes  these  alternatives.  In  fact,  a  recent  DoD  study  submitted  to 
Congress  even  identified  the  lifecycle  costs  for  each  of  these  three  alternatives  for  replacing  the 
forty-year-old  engines  on  one  of  America’s  most  successful  combat  aircraft:  Joint  STARS: 

Option  A:  Outright  Purchase:  $  1.0  Billion 

Option  B:  Purchase  Over  Time:  $  1.2  Billion 

Option  C:  Do  Nothing:  $  1.5  Billion 

Any  reasonable  person  would  quickly  understand  that  Option  A  is  the  best  choice.  And 
if  for  some  reason  Option  A  is  not  possible,  Option  B  is  the  next  best  alternative.  Clearly, 

Option  C  is  the  worst  choice. 

But  when  viewed  through  the  prism  of  Federal  budget  policy,  the  logic  gets  twisted  in 
this  way:  Because  Option  B  costs  more  than  Option  A,  Option  B  must  be  eliminated  from  any 
further  consideration.  So  the  choice  is  between  Option  A  and  Option  C.  But  history  shows 
Option  A  is  not  a  realistic  possibility,  since  the  DoD  usually  has  more  urgent  priorities  than 
replacing  engines  that,  after  all,  still  work.  So  Option  C  is  the  end  result. 

Privately,  everyone  agrees  that  Option  A  is  not  likely  to  happen.  Like  a  frog  in  boiling 
water,  Federal  decision-makers  never  really  feel  the  pain  of  creeping  obsolescence  in  weapons 
systems.  As  a  result,  even  the  most  economically  sound  upgrades  rarely  compete  successfully 
for  budget  dollars  against  higher  priorities.  Nor  is  this  necessarily  a  bad  decision.  Imagine 
having  to  choose  between  upgrading  Humvees  with  either  new  armor  against  roadside  bombs 
or  a  new  engine.  New  armor  will  save  lives  right  away,  so  it  should  (and  does)  get  priority. 

This  example  is  representative  of  the  difficult  choices  made  every  day.  The  point  is  that 
even  if  we  assume  that  DoD  and  Congress  sort  out  these  priorities  properly,  shouldn’t  Option  B 
at  least  remain  on  the  table  for  consideration?  If  “paid-from-savings”  contracts  are  a  legitimate 
tool  for  upgrading  the  rear  echelon,  why  shouldn’t  the  tool  be  available  on  the  front  line? 

Several  years  ago,  a  bipartisan  group  of  Senators  and  Congressmen  ranging  from 
Senators  Sue  Collins  to  Pat  Roberts  to  Hillary  Clinton  introduced  legislation  that  would  allow  the 
DoD  to  use  Energy  Savings  Performance  Contracts  to  upgrade  combat  aircraft,  ships  and 
vehicles.  This  was  a  bill  (S.  2318  /  H.R.  3339)  that  appealed  to  hawks,  environmentalists  and 
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anti-government  waste  advocates  equally.  There  was  no  apparent  opposition  until  the  CBO  and 
OMB  accountants  successfully  killed  the  effort. 

The  CBO  asserted  that  Energy  Saving  Performance  Contracts  outlays  are  real,  but  their 
savings  are  speculative — even  though  they  are,  by  law,  a  mathematical  identity.  The  House 
Energy  Committee  Chairman,  Joe  Barton,  called  the  CBO’s  reasoning  “absurd,”  and  Senate 
Energy  Committee  Chairman  (and  former  Budget  Committee  Chairman)  Pete  Domenici  stated: 
“CBO  views  these  contracts  as  outlays  by  the  federal  government.  The  truth  is  that  these 
contracts  cost  the  government  nothing.” 

But  because  of  the  CBO,  Congress  was  barely  able  to  muster  the  votes  to  reauthorize 
these  contracts  for  use  in  upgrading  military  buildings,  gagging  down  a  $2.8  billion  “score”  from 
the  CBO.  Tragically  for  our  warfighters  and  taxpayers,  the  CBO’s  Alice-in-Wonderland 
accounting  estimated  an  expansion  of  these  contracts  for  use  on  combat  systems  at  about  $15 
billion.  This  “cost”  was  too  much  for  anyone  to  ignore,  but  not  enough  for  anyone  to  engage  in  a 
political  firefight  with  the  CBO.  So,  the  Energy  Act  of  2005  was  passed  with  Energy  Savings 
Performance  Contracts  for  military  buildings  fully  reauthorized,  but  nothing  authorized  for 
battlefield  assets. 

The  great  irony  is  that  the  CBO’s  scoring  policy  makes  it  painful  to  save  a  little  taxpayer 
money  and  impossible  to  save  a  lot. 

Contributing  to  and  applauding  this  perverse  outcome  is  the  OMB,  the  accountants  for 
the  Executive  Branch,  including  the  DoD.  To  their  credit,  the  OMB  believes  that  Energy  Savings 
Performance  Contracts  cost  the  government  nothing.  But  the  OMB  went  out  of  its  way  to 
disparage  using  this  proven  tool  to  upgrade  combat  systems.  The  reason?  Only  that,  “it  is 
inconsistent  with  Federal  fiscal  and  procurement  policies.”  No  kidding... 

In  a  less  dangerous  world,  wasting  defense  dollars  by  equipping  our  warfighters  with 
second-rate  equipment  that  costs  more  than  first-rate  alternatives  would  simply  be  irrational. 

But  for  the  foreseeable  future,  irrational  budget  policies  will  continue  to  have  very  real  and 
dangerous  consequences  for  the  brave  Americans  we  send  into  battle. 

Warfighters  deserve  more  respect  than  these  accountants  give  them.  And  the  Bush 
Administration  and  Congress  should  get  serious  about  where  their  priorities  are:  with  the 
taxpayers  and  warfighters,  or  with  the  accountants. 

Background 

The  original  statute  creating  Energy  Savings  Performance  Contracts  (ESPCs)  was  the 
Energy  Policy  Act  of  1 992,  Public  Law  102-486,  enacted  October  24,  1992,  codified  at  42  USC 
8287.  The  CBO  scored  the  ESPC  provisions  of  the  Energy  Policy  Act  of  1992  at  zero. 

The  sunset  date  of  the  original  statute  was  amended  to  become  October  1 , 2003,  by  the 
Energy  Conservation  Reauthorization  Act  of  1998  (Public  Law  105-388,  enacted  November  13, 
1998).  Again,  the  CBO  scored  the  ESPC  provisions  of  the  Energy  Conservation 
Reauthorization  Act  of  1998  at  zero. 

The  CBO  reversed  this  decade-old  policy  of  scoring  ESPCs  at  zero  on  April  7,  2003  in 
their  Cost  Estimate  for  H.R.  1346,  in  which  the  CBO  stated: 
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Currently,  federal  agencies  can  enter  into  an  ESPC,  a  specific  type  of  long-term 
contract,  for  the  purchase  of  energy  efficiency  equipment,  such  as  new  windows  and 
lighting.  Using  such  equipment  can  reduce  the  energy  costs  for  a  facility.  When  using 
an  ESPC,  the  savings  from  reduced  energy  bills  are  used  to  pay  for  the  purchase  of  the 
new  equipment  over  several  years.  The  commitment  to  make  such  payments  is  made 
when  the  ESPC  is  entered  into.  Thus,  consistent  with  governmentwide  accounting 
principles,  CBO  believes  that  the  budget  should  reflect  that  commitment  as  new 
obligations  at  the  time  that  an  ESPC  is  signed.  Currently,  agencies  can  use  ESPCs  to 
purchase  new  equipment  over  a  25-year  period  without  an  appropriation  for  the  full 
amount  of  the  purchase  price. 

DOE  estimates  that  agencies  entered  into  ESPCs  valued  over  $800  million  since  1988. 
CBO  estimates  that,  because  the  federal  building  inventory  is  aging,  those  contracts 
would  continue  to  be  used  over  time  at  roughly  the  same  rate  currently  used,  or  $75 
million  in  2004  and  increasing  after  that.  Thus,  we  estimate  that  extending  the 
authorization  for  ESPCs  would  increase  direct  spending  by  about  $64  million  in  2004 
and  $1.1  billion  over  the  2004-2013  period. 

This  Cost  Estimate  was  prepared  by  Lisa  Cash  Driskill  and  approved  by  Peter  H. 
Fontaine,  Deputy  Assistant  Director  for  Budget  Analysis. 

The  following  day,  on  April  8,  2003,  the  CBO  published  their  Cost  Estimate  for  H.R.  6, 
which  states: 

Energy  Savings  Performance  Contracts  (ESPCs).  Section  11006  of  H.R.  6  would 
provide  permanent  authorization  to  use  ESPCs  and  would  expand  their  use.  The 
expansion  would  allow  agencies  to  use  an  ESPC  to  construct  replacement  buildings  by 
committing  to  pay  private  contractors  a  portion  of  the  budget  savings  expected  from 
reduced  operations,  maintenance,  and  energy  costs  at  such  new  buildings.  CBO 
estimates  that  this  provision  would  cost  $75  million  in  2004,  $879  million  over  the  2004- 
2008  period,  and  $2.8  billion  over  the  next  10  years. 

Again,  this  section  of  the  Cost  Estimate  for  H.R.  6  was  prepared  by  Lisa  Cash  Driskill 
and  approved  by  Peter  H.  Fontaine,  Deputy  Assistant  Director  for  Budget  Analysis. 

In  sum,  the  CBO  recognized  that  ESPCs  cost  the  government  nothing  and  scored  ESPC 
authorization  and  reauthorization  at  zero  in  1992  and  1998,  respectively.  In  2003,  with  no 
meaningful  explanation,  CBO  reversed  this  policy  and  scored  ESPCs  as  direct  spending. 

For  its  part,  the  Office  of  Management  and  Budget  (OMB)  rejects  the  CBO’s  new  ESPC 
scoring  policy  and  continues  to  score  ESPCs  as  budget-neutral  except  for  termination  liability, 
which  is  scored  only  if  and  when  such  termination  actually  occurs.  This  OMB  policy  is  set  forth 
in  the  July  25,  1998,  OMB  memorandum  Federal  Use  of  Energy  Savings  Performance 
Contracting  (Retrieved  from  http://www.whitehouse.gov/omb  /memoranda/m98-1 3.pdf). 

This  policy,  originally  established  in  the  Clinton  Administration,  was  reaffirmed  as  the 
policy  of  the  Bush  Administration  by  the  Secretary  of  Energy  in  letters  to  the  Chairmen  of  the 
House  and  Senate  Energy  Committees,  dated  April  8,  2004,  in  which  Secretary  Abraham 
states: 
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The  legislation  itself  extending  ESPC  authority  is  considered  budget  neutral  and  does 
not  require  additional  resources,  as  the  Office  of  Management  and  Budget  classifies  all 
budget  authority  and  outlays  for  ESPCs  as  absorbing  discretionary  resources.  However, 
ESPCs  actually  save  the  government  money,  because  the  upfront  costs  of  ESPC 
efficiency  improvements  are  recovered  through  the  energy  savings  that  result. 

Moreover,  payments  to  the  contractors  are  contingent  upon  realizing  a  guaranteed 
stream  of  future  cost  savings. 


Improved  energy  efficiency  and  conservation  at  Federal  facilities  is  an  important 
component  of  this  Administration’s  commitment  to  the  cost-effective  use  of  public  dollars 
and  protection  of  the  environment.  The  Administration  urges  Congress  to  act  quickly  to 
the  authorization  of  this  important  program. 

Thus,  the  consistent  position  of  the  Executive  Branch  through  both  the  Clinton  and 
current  Bush  Administrations  is  that  ESPCs  should  be  scored  at  zero.  The  reasoning  for  this  is 
compelling  since,  contrary  to  CBO’s  claim  that  the  government’s  “commitment  to  make  such 
payments  is  made  when  the  ESPC  is  entered  into,”  the  government’s  commitment  to  make 
payments  under  an  ESPC  only  is  made  when,  and  to  the  extent,  savings  are  achieved  in  each 
year  of  the  ESPC.  This  is  set  forth  explicitly  in  42  USC  8287  (B),  which  states: 


Aggregate  annual  payments  by  an  agency  to  both  utilities  and  energy  savings 
performance  contractors,  under  an  energy  savings  performance  contract,  may  not 
exceed  the  amount  that  the  agency  would  have  paid  for  utilities  without  an  energy 
savings  performance  contract  (as  estimated  through  the  procedures  developed  pursuant 
to  this  section)  during  contract  years. 

Thus,  contrary  to  the  CBO’s  assertion  that  its  new  ESPC  scoring  policy  is,  “consistent 
with  governmentwide  accounting  principles,”  no  other  entity  within  the  federal  government  has 
ever  accepted  the  CBO’s  policy.  Just  the  opposite  is  true:  Not  scoring  ESPC  payments  is,  and 
always  has  been,  the  governmentwide  accounting  principle. 

Economic  Logic  Compels  ESPCs  Scoring  at  Zero 

The  fundamental  economic  basis  for  not  scoring  ESPCs  is  that  the  opportunity  cost  of  an 
ESPC  (i.e.,  the  government’s  next  best  alternative  to  the  ESPC)  is  to  continue  to  pay  (what  by 
law  must  be)  a  higher  amount  for  ongoing  energy  and  maintenance  costs  on  the  aging 
infrastructure  that  the  ESPC  would  upgrade. 

The  fact  that  ESPCs  must  reduce  pre-existing  government  obligations  makes  ESPCs 
unique  among  all  financing  vehicles  available  to  the  government.  Scoring  ESPC  payments 
without  deducting  the  higher  payments  the  government  would  otherwise  make  results  in  double 
counting  of  the  true  scope  of  the  government  obligation. 

Put  another  way,  if  a  government  obligation  of  $100  is  replaced  with  a  lesser 
government  obligation  of  $90,  the  correct  scoring  is  not  $190.  While  one  could  argue  that  the 
proper  score  in  this  case  is  $90,  proponents  of  ESPCs  only  ask  that  the  more  conservative 
amount  of  $100  be  recognized  as  a  complete  offset  to  the  original  $100. 
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The  fact  is  that  using  an  ESPC  cannot  increase  Federal  obligations.  At  worst,  the 
Federal  obligation  remains  at  the  same  level  it  would  have  been  if  no  ESPC  were  used.  It  is  this 
worst-case  scenario  that  produces  a  zero  score  in  any  rational  budget  treatment. 

Accounting  Logic  Compels  ESPCs  Scoring  at  Zero 

Recognizing  that  accounting  principles  sometimes  differ  from  their  underlying  economic 
theory,  it  is  worth  reviewing  ESPC  scoring  through  a  purely  accounting  prism.  In  the  most 
general  sense,  Federal  accounting  divides  Federal  spending  into  Operations  &  Maintenance 
(O&M)  accounts  and  Capital  accounts.  In  a  classic  Federal  acquisition,  an  upgrade  is 
purchased  by  an  agency  using  Capital  appropriations  provided  for  that  specific  purpose  by 
Congress,  almost  always  without  any  statutory  offset  requirements.  In  such  a  case,  the  entire 
amount  of  that  Capital  appropriation  is  properly  scored  as  direct  spending.  Linking  this  to  its 
underlying  economic  justification,  it  can  be  said  that  Congress  could  spend  that  appropriation  on 
anything  else;  thus,  its  opportunity  cost  is  the  full  amount  of  the  Capital  appropriation. 

An  ESPC  is  fundamentally  different  from  such  a  classic  Federal  acquisition,  precisely 
because  it  only  draws  on  the  existing  stream  of  the  applicable  O&M  account  over  the  term  of  the 
ESPC.  The  key  factor  that  compels  a  zero-score  accounting  treatment  is  that  the  ESPC  cannot 
ever  draw  more  from  that  O&M  account  than  would  have  been  drawn  if  the  ESPC  had  not  been 
executed.  Moreover,  the  ESPC  can  never  create  Federal  obligations  from  any  Capital  account. 
Again,  linking  this  to  its  underlying  economic  justification,  Congress  could  not  spend  that  portion 
of  the  O&M  account  appropriation  on  anything  else,  since  it  would  be  spent  on  fuel  and 
maintenance  for  the  aging  asset  in  the  absence  of  an  ESPC;  thus,  the  opportunity  cost  of  an 
ESPC  is  spending  the  same  (or  greater)  amount  of  O&M  appropriation. 

In  sum,  unless  there  is  a  contract  termination,  ESPCs  cannot  ever  lead  to  an  increase  in 
the  amount  of  money  that  Congress  would  have  otherwise  appropriated  for  any  O&M  or  Capital 
account. 

To  be  clear,  termination  of  an  ESPC  can  trigger  a  Federal  obligation  that  would  exceed 
the  normal  O&M  account  funding  stream.  Congress  recognized  this  when  it  created  ESPCs  in 
1992  and  explicitly  set  forth  how  such  an  event  should  be  scored.  This  provision,  codified  in  42 
USC  8287,  states: 

1.  A  Federal  agency  may  enter  into  a  multiyear  contract  under  this  subchapter  for  a  period 
not  to  exceed  25  years,  without  funding  of  cancellation  charges  before  cancellation,  if— 

(i)  such  contract  was  awarded  in  a  competitive  manner  pursuant  to  subsection 
(b)(2)  of  this  section,  using  procedures  and  methods  established  under  this  subchapter; 

(ii)  funds  are  available  and  adequate  for  payment  of  the  costs  of  such  contract  for 
the  first  fiscal  year; 

(Hi)  30  days  before  the  award  of  any  such  contract  that  contains  a  clause  setting 
forth  a  cancellation  ceiling  in  excess  of  $10,000,000,  the  head  of  such  agency  gives 
written  notification  of  such  proposed  contract  and  of  the  proposed  cancellation  ceiling  for 
such  contract  to  the  appropriate  authorizing  and  appropriating  committees  of  the 
Congress;  and 
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(iv)  such  contract  is  governed  by  part  17.1  of  the  Federal  Acquisition  Regulation 
promulgated  under  section  421  of  title  41  or  the  applicable  rules  promulgated  under  this 
subchapter. 

Thus,  the  proper  accounting  treatment  to  provide  for  possible  termination  is  not  a  matter 
for  debate  or  interpretation;  it  is  explicitly  established  by  Congress  in  Federal  law.  To  the  extent 
that  the  CBO  justifies  its  radical  scoring  of  ESPCs  by  citing  termination  liability  exposure,  it  is 
contrary  to  the  consensus  of  Congress  expressed  in  this  statute. 

Moreover,  this  law  makes  tremendous  sense  and  reaches  the  proper  economic  result. 
Accounting  for  termination  costs  is  clearly  appropriate  when  a  termination  occurs.  But  since  no 
one  knows  the  future  with  certainly,  the  termination  liability  amount  should  be  discounted  by  the 
likelihood  of  a  termination  actually  occurring.  Historical  data  shows  that  less  than  1  %  of  all 
Federal  contracts  are  terminated,  either  for  government  convenience  or  contractor  default. 
Twelve  years  of  experience  shows  that  ESPCs’  rate  of  termination  is  much  better  than  these 
government-wide  averages. 

What  Does  It  Matter? 

This  CBO  scoring  policy  has  a  tremendously  adverse  impact  and  is  against  Federal 
Government  interests.  Simply  put,  this  CBO  scoring  policy  undermines  the  original  purpose  for 
which  Congress  intended  ESPCs,  which  is  to  permit  agencies  to  introduce  rational  economic 
upgrades  that  permanently  reduce  net  costs  to  the  taxpayer. 

This  CBO  scoring  policy  further  undermines  the  beneficial  expansion  of  ESPCs  to  the 
non-installation  applications,  as  was  provided  for  in  H.R.  3339  and  S.  2318.  These  upgrades  to 
combat  aircraft,  tanks  and  ships  are  where  the  majority  of  benefits  to  the  Federal  government 
would  accrue.  Because  ESPCs  in  this  application  also  increase  the  combat  range  and  reliability 
of  military  forces,  the  adverse  impact  of  this  CBO  policy  will  result  in  American  forces  being  sent 
into  battle  with  less  than  the  best  available  weapons  and  support  systems  per  dollar  spent. 

While  a  healthy  debate  can  always  be  made  on  what  is  the  right  level  of  military 
spending,  it  is  unconscionable  that  once  that  level  is  set,  we  do  not  provide  our  warfighters  the 
most  powerful  capability  that  this  amount  of  money  can  buy. 

In  a  Perfect  World 

While  the  immediate  solution  would  be  to  reverse  the  CBO’s  scoring  policy  on  paid-from- 
savings  contracting,  there  is  a  more  proactive  approach  that  should  be  considered.  The  best 
solution  is  to  adopt  more  responsible  policy  along  the  following  lines: 

a.  The  overall  acquisition  process  should  result  in  the  compilation  and  maintenance  of  a  list 
of  acquisitions  that  make  economic  and  operational  sense,  using  rational  and 
intellectually  honest  lifecycle  cost-benefit  analysis. 

b.  Pay  for  as  many  of  these  acquisitions  as  possible  with  the  limited  capital  appropriations 
available  each  year.  Regardless  of  how  the  projects  are  prioritized  (economic  priorities 
will  often  lose  out  to  political,  strategic  and  tactical  priorities),  at  the  end  of  the  process 
there  will  be  unfunded  but  worthy  acquisitions. 

c.  For  each  unfunded  but  desired  acquisition,  determine  if  the  potential  cost  of  the 
acquisition  would  be  offset  by  a  corresponding  savings  generated  by  the  acquisition. 
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d.  In  cases  where  such  an  offset  is  sufficient  to  cover  a  multi-year  amortization  of  the 
acquisition,  Federal  agencies  should  be  encouraged  to  use  multi-year  contracting 
authority  to  match  the  rate  of  new  spending  for  the  upgrade  to  the  rate  of  new  savings 
generated  by  the  upgrade. 

e.  Require  agencies  to  consider  total  savings  to  the  US  T reasury,  not  just  savings  in  their 
corner  of  the  Federal  government.  Rational  accounting  rules  for  a  national  government 
should  not  encourage  tribalism. 

Finally,  it  should  be  recognized  that  some  upgrades  do  eventually  get  funded  through 
the  traditional  acquisition  process.  Defenders  of  the  status  quo  are  quick  to  point  to  these 
examples  as  evidence  that  the  traditional  system  works.  The  flaw  in  this  logic  is  that  years  of 
potential  savings  are  lost  forever  while  upgrades  wait  in  line  for  full  funding.  The  net  effect  of 
this  folly  is  that  taxpayers  and  warfighters  are  cheated  for  the  sake  of  blind  obedience  to  a 
bureaucratic  system  that  serves  no  one  but  itself. 
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Abstract 

This  research  investigates  procurement  scoring  and  the  Department  of  Defense’s  (DoD) 
use  of  alternative  financing  methods,  such  as  leases  and  public-private  ventures.  One  of  the 
major  impediments  to  using  alternative  forms  of  procurement  financing  for  acquiring  defense 
capabilities  is  in  the  budgetary  treatment,  or  “scoring,”  of  these  initiatives  by  the  Congressional 
Budget  Office  (CBO),  the  Office  of  Management  and  Budget  (OMB)  and  the  congressional 
Budget  Committees.  The  current  scoring  policy  that  has  been  applied  to  many  initiatives 
essentially  negates  the  financial  advantage  for  using  alternative  forms  of  financing.  Therefore, 
this  research  examines  existing  policies  and  their  adherence  to  statutes  and  the  role  of  the 
various  government  organizations  and  committees  in  actual  recording  of  obligations  and  outlays 
related  to  financing  alternatives  used  by  federal  agencies.  Preliminary  evidence  suggests  that 
this  emerging  area  has  major  importance  for  future  DoD  acquisitions  in  a  resource-constrained 
environment.  Included  are  recommendations  for  changes  in  budgetary  scoring  that  encompass 
the  full  scope  of  federal  obligations  and  expenditures  while  promoting  efficient,  more  rapid  and 
fiscally  responsible  acquisitions. 

Executive  Summary 

Due  to  the  increasing  fiscal  pressure  caused  by  the  Global  War  on  Terror  (GWOT)  and 
the  growing  burden  of  entitlement  programs,  the  Department  of  Defense  (DoD)  must  consider 
alternative  forms  of  financing,  including  leases  and  public-private  partnerships  (PPPs),  to  fund 
necessary  programs.  This  research  examines  the  budgetary  treatment,  or  scoring,  of  these 
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financial  arrangements  by  the  Office  of  Management  and  Budget  (OMB),  the  Congressional 
Budget  Office  (CBO)  and  the  House  and  Senate  Budget  Committees.  Every  congressional 
legislation  must  be  scored  in  accordance  with  the  federal  budget  process.  Scoring  legislation  is 
the  process  of  tracking  budget  authority,  projecting  future  federal  outlays  based  on  the  budget 
authority,  and  recording  the  actual  obligations  and  outlays  in  budget  execution.  The  scoring 
process  can  greatly  affect  a  bill’s  ability  to  be  passed  based  on  the  financial  considerations 
made  by  the  CBO,  OMB  or  Congressional  Budget  Committees. 

This  research  introduces  the  current  applications  of  leasing  and  PPPs  in  the  public  and 
private  sectors.  Additionally,  an  in-depth  analysis  of  the  current  scoring  process  conducted  by 
the  CBO,  OMB  and  the  Budget  Committees  will  be  discussed.  These  government  bodies 
represent  the  executive  and  legislative  authorities  for  financing.  This  analysis  will  be  applied  to 
three  case  studies,  the  budgetary  treatment  of  Energy  Savings  Performance  Contracts  (ESPC), 
and  two  cases  involving  the  use  of  PPPs  in  the  Operation  and  Maintenance  of  Military  Family 
Housing. 

Current  scoring  and  general  federal  budget  policies  negate  the  advantages  of  using 
alternative  forms  of  financing  such  as  leasing  and  PPPs.  Therefore,  they  are  not  used  in  the 
acquisition  of  major  defense  assets,  even  though  they  have  proven  to  generate  substantial 
benefits  for  the  private  sector  by  providing  greater  flexibility  in  financing,  encouraging 
innovation,  reducing  risks,  and  saving  time  and  money  on  projects.  This  research  identifies  the 
scoring  policies  of  both  the  OMB  and  CBO  and  recommends  a  revised  scoring  policy  that 
applies  financial  responsibility  as  well  as  fair  treatment  of  the  advantages  of  these  initiatives. 
The  end  goal  is  not  to  develop  a  solution  that  will  revamp  the  current  budget  process,  but  to 
provide  a  policy  that  will  to  secure  funding  for  needed  defense  programs  while  satisfying  the 
requirements  of  fiscal  accountability. 

Introduction 

The  conventional  method  of  procurement  for  major  government  acquisitions  is  full-cost 
and  up-front  funding.  Full-cost  funding  means  that  appropriations  must  be  sufficient  to  cover  a 
capital  project  prior  to  any  obligations  being  incurred.  In  other  words,  the  full  cost  of  the 
program  must  be  accounted  for  in  the  first  year  of  obligation.  The  policy  provides  transparency 
in  the  budget;  in  other  words,  all  programs  are  scored  in  the  same  manner  so  that  proper  cost 
comparisons  can  be  made  between  projects.  Additionally,  full  funding  secures  funds  for  the 
total  cost  of  the  project,  minimizing  the  need  for  additional  funding  in  the  future. 

Full-cost  funding  forces  military  departments  to  analyze  each  project’s  cost  and  benefits 
throughout  its  life.  It  ensures  that  future  congressional  action  is  not  required  to  pay  for  previous 
congressional  spending  decisions.  Also,  full-cost  funding  empowers  program  managers  to  be 
responsible  for  time,  schedule,  and  cost  parameters  of  a  project.  While  full-cost  funding 
certainly  has  its  benefits,  the  policy  can  cause  major  fluctuations  in  appropriations  that  might 
eliminate  the  ability  to  fund  a  justified  program.  Particularly  with  large  acquisitions,  full-cost 
funding  consumes  a  large  portion  of  a  military  department's  available  funding  resources,  thus 
reducing  the  funds  available  for  other  programs. 

With  the  growing  cost  of  the  Global  War  on  Terror  (GWOT),  particularly  with  Operations 
and  Maintenance  (O&M)  and  growing  technology  costs,  the  Department  of  Defense  (DoD)  is 
under  increasing  pressure  to  secure  funding  for  large  capital  projects  with  a  smaller  percentage 
of  the  budget  designated  for  new  procurement  of  combat  capability.  Therefore,  alternative 
financing  arrangements,  including  but  not  limited  to  incremental  funding,  operating  leases,  out- 
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leases,  share-in-savings  contracts,  and  public  private  partnerships  (PPPs),  have  attracted 
interest  as  potential  alternative  financing  methods.  The  potential  advantages  and 
disadvantages  of  these  financing  methods  will  be  analyzed,  along  with  the  scoring  methods  that 
determine  their  cost. 

The  Office  of  Management  and  Budget  (OMB)  for  the  Executive  Branch,  the 
Congressional  Budget  Office  (CBO)  and  Congressional  Budget  Committees  for  the  Legislative 
Branch  have  the  collective  responsibility  for  determining  the  benefits  and  costs  for  DoD 
appropriations  bills.  This  power,  as  dictated  in  the  Congressional  Budget  Act  of  1974, 
empowers  these  agencies  as  the  official  “scorekeepers,”  who  determine  the  actual  cost  of 
programs  and  their  relationship  to  the  overall  National  Military  Strategy.  However,  operating 
under  the  same  scoring  guidelines,  divergent  scoring  results  arise  between  the  OMB,  CBO  and 
the  Budget  Committees  based  on  different  interpretations  of  the  scoring  principles. 

This  research  focuses  on  the  benefits  of  alternative  financing  and  the  scoring  of  these 
benefits  from  alternative  financing  agreements.  With  increasing  fiscal  pressure,  these  methods 
are  necessary  to  provide  funding  for  needed  acquisitions.  In  the  GWOT  and  the  more  hostile 
world  in  which  we  live,  the  Warfighter  cannot  wait  until  the  next  budget  cycle  for  the  equipment 
needed  to  complete  the  assigned  mission.  A  revised  scoring  policy  is  recommended  to  permit 
the  DoD  to  fund  additional  procurement  projects  within  the  same  budget  constrains,  using 
fiscally  sound,  generally  accepted  accounting  principles. 

Background  into  Scoring 

The  term  “scoring”  describes  the  process  in  which  the  CBO  and  OMB  estimate  the 
budget  authority  required  by  proposed  legislation.  Budget  authority  is  the  authority  provided  by 
law  to  incur  financial  obligations  that  will  result  in  monetary  outlays  (OMB,  2006,  June). 
Scorekeeping  determines  in  a  dollar  amount  the  budget  effects  of  legislation  and  forecasts 
future  outlays  needed  to  fund  a  program.  The  “scorekeepers”  consist  of  the  Congressional 
Budget  Committees,  the  CBO,  and  the  OMB.  The  scoring  process  and  principles  used  by  these 
entities  greatly  impact  the  scored  “cost”  of  a  program  and,  consequently,  the  ability  of  the 
legislation  to  be  passed  by  Congress.  The  current  scoring  guidelines  greatly  limit  the 
advantages  of  alternative  financing  arrangements  that  attempt  to  draw  on  private-sector 
expertise  and  funding.  This  section  analyzes  the  scoring  rules  that  apply  to  lease,  lease- 
purchase,  and  capital  acquisition  arrangements  and  addresses  the  disadvantages  of  the  current 
guidelines.  It  also  provides  a  background  into  Energy  Savings  Performance  Contracting 
(ESPC). 

The  scoring  guidelines  contained  in  OMB  Circular  A-1 1  embody  two  fundamental 
principles  of  federal  budgeting: 

Federal  commitments  should  be  recognized  up-front  in  the  budget;  at  the  time  those 
commitments  are  made. 

Budget  should  be  comprehensive,  capturing  all  financial  activities  of  the  federal  budget 
(President’s  Commission  on  Budget  Concepts,  1967,  October). 

These  principles  form  a  policy  known  as  full-funding  that  requires  agencies  to  request  all 
funding  for  a  project  up-front.  Prior  to  1991,  the  budget  authority  and  outlays  for  most  leases 
were  recognized  annually  over  the  lease  term  in  the  form  of  annual  lease  payments.  This  policy 
allowed  agencies  to  acquire  an  asset  without  Congress’  consent  for  the  full  funding  of  the  asset. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  378- 


In  1991,  new  guidelines  issued  by  the  OMB  scored  capital  leases  and  lease-purchases 
as  up-front  and  requiring  full  funding.  The  policy  is  designed  to  force  decision-makers  to 
determine  the  entire  cost  of  a  project  prior  to  approving  the  legislation.  The  up-front  funding 
allows  for  greater  Congressional  control  over  appropriations  and  also  allows  Congress  to 
evaluate  multiple  pieces  of  legislation  on  a  cost  basis.  This  “transparency”  provides  Congress 
with  a  “standard”  with  which  to  monitor  the  spending  of  both  individual  agencies  and  the  entire 
federal  government  on  an  annual  basis. 

Full  funding  also  better  aligns  Congressional  budget  estimates  with  the  Anti-deficiency 
Act  (31  USC  1 341 ),  which  prohibits  the  government  from  entering  into  obligations  for  the 
payment  of  money  before  an  appropriation  is  made,  unless  authorized  by  law.  Full  funding  is  a 
policy  rather  than  a  law,  which  means  that  the  interpretation  of  the  policy  can  impact  the 
budgetary  treatment  of  a  program.  Whether  an  asset  is  acquired  via  direct  purchase,  lease,  or 
through  a  combination  of  the  two,  scoring  rules  are  currently  biased  towards  full  funding.  These 
financial  arrangements,  as  well  an  analysis  of  the  impact  of  the  current  scoring  rules,  will  be 
addressed  below. 

Federal  Budget  Principles:  Purchases,  Leases,  and  Alternative 
Financing 

Direct  Purchases 

A  simple  example  of  budget  scoring  is  an  outright  purchase:  the  government’s  budget 
commitment  is  the  purchase  price  of  the  asset.  Budget  authority  is  assessed  equal  to  the 
purchase  price  of  the  asset  at  the  time  when  authority  is  received  to  acquire  the  asset.  Outlays 
are  then  recorded  when  actual  cash  payments  are  made  to  the  seller  (CBO,  2003).  Outright 
purchases  can  be  financed  through  borrowing  at  a  low  interest  rate  from  the  US  Treasury  (e.g., 
Treasury  Bills  sold  publicly),  whereas  leases  require  a  higher  interest  rate  due  to  private-sector 
financing.  The  scoring  policy  does  not  account  for  several  inherent  costs  of  directly  purchasing 
an  asset.  Full-funding  an  asset  requires  the  government  to  assign  a  larger  proportion  of  the 
available  budget  authority  to  the  asset,  leaving  less  budget  authority  available  for  other  assets  in 
any  given  fiscal  year.  Under  this  policy,  a  larger  opportunity  cost  exists  as  decision-makers 
must  often  decide  between  two  mutually  exclusive  programs  rather  than  funding  both.  Military 
Departments  must  often  delay  or  cancel  large  capital  investments  that  will  offer  better 
performance  and  lower  long-term  costs  to  realize  short-term  savings.  Benefits  generated  by 
these  large  projects  are  realized  over  several  years,  whereas  the  costs  must  be  realized  up¬ 
front.  Outright  purchasing  may  force  elected  officials  to  choose  between  two  or  more  justifiable 
programs,  when  both  programs  could  be  funded  through  other  means. 


Leasing 

To  distribute  the  acquisition  cost  of  an  asset  over  its  years  of  use,  the  government  has 
the  ability  to  lease  the  asset,  or  in  some  cases,  to  enter  into  a  partnership  with  private 
companies  to  acquire  the  asset.  A  simple  lease  arrangement  involves  an  owner  (lessor)  renting 
the  use  of  an  asset  to  another  party  (lessee).  For  example,  the  rental  of  an  automobile  from 
Avis  implies  no  ownership.  However,  leases  can  be  structured  in  an  almost  limitless  number  of 
complex  arrangements  in  which  all  terms  are  negotiable,  and  third-party  financing  may  be 
involved. 
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To  limit  the  discussion,  leases  within  government  are  placed  into  four  broad  categories: 
operating  leases,  capital  leases,  and  lease  purchases  with  or  without  substantial  private  risk. 
The  distinction  between  the  different  lease  types  determines  how  the  CBO,  OMB  and  the 
Budget  Committees  score  budget  authority  for  legislation.  Each  lease  category  is  discussed 
below. 


To  be  considered  an  “operating”  lease,  a  lease  must  satisfy  the  following  stringent 

criteria: 


Ownership  of  the  asset  remains  with  the  lessor  during  the  term  of  the  lease  and  is  not 
transferred  to  the  Government  at  or  shortly  after  the  end  of  the  lease  period. 

The  lease  does  not  contain  a  bargain-price  purchase  option. 

The  lease  term  does  not  exceed  75%  of  the  estimated  economic  lifetime  of  the  asset. 

The  present  value  of  the  minimum  lease  payments  over  the  life  of  the  lease  does  not 
exceed  90%  of  the  fair  market  value  of  the  asset  at  the  inception  of  the  lease. 

The  asset  is  a  general-purpose  asset  rather  than  a  special-purpose  asset  for  the 
Government  and  is  not  built  to  unique  specification  for  the  Government  as  lessee. 

There  is  a  private-sector  market  for  the  asset. 

Risks  of  ownership  of  the  asset  should  remain  with  the  lessor  (OMB,  2006,  June,  part  8- 
appendices). 

Any  lease  not  satisfying  these  stringent  criteria  will  be  viewed  as  a  “capital”  lease. 
Requirements  5  and  6  (the  need  for  a  private-sector  market  for  the  asset  and  the  requirement 
for  the  asset  to  be  general  purpose)  essentially  eliminate  operating  leasing  for  military 
equipment  procurement. 

In  both  operating  and  capital  leases,  ownership  remains  with  the  lessor  and  is  not 
transferred  to  the  government  at  the  conclusion  of  the  lease  period.  In  contrast,  lease-purchase 
arrangements  allow  ownership  of  the  asset  to  be  transferred  (GAO,  1997).  Determination  of 
risk  is  another  crucial  determination  in  the  budget-scoring  process.  In  OMB  Circular  A-1 1 ,  risk 
is  defined  in  relation  to  the  government-specific  characteristics  of  the  project.  The  more 
governmental  the  project,  the  greater  amount  of  risk  is  assigned  to  the  government.  Legislation 
and  lease-purchases  use  the  following  criteria  to  determine  the  amount  of  risk  borne  by  the 
government. 

There  should  be  no  provision  of  Government  financing  and  no  explicit  Government 
guarantee  of  third-party  financing. 

Risks  of  ownership  of  the  asset  should  remain  with  the  lessor  unless  the  Government 
was  at  fault  for  such  losses. 

The  asset  should  be  a  general-purpose  asset  rather  than  for  a  special  purpose  of  the 
Government  and  should  not  be  built  to  unique  specification  for  the  Government  as  lessee. 

There  should  be  a  private-sector  market  for  the  asset. 
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The  project  should  not  be  constructed  on  Government  land. 

The  ambiguity  of  these  guidelines  demonstrates  that  they  are  more  “policies”  rather  than 
scoring  “rules.”  The  interpretation  of  these  guidelines  has  been  the  source  of  frustration  for 
many  private-public  partnership  initiatives. 

The  budgetary  treatment  of  the  four  categories  of  lease  arrangements  is  summarized  in 
Table  1  below. 


Transaction 

Budget  Authority 

Outlays 

Lease-purchase 
Without  Substantial 
Private  Risk 

Amount  equal  to  asset  cost  recorded 
up-front;  amount  equal  to  imputed 
interest  costs  recorded  on  an  annual 
basis  over  the  lease  period 

Amount  equal  to  asset  cost  scored  over  the 
construction  period  in  proportion  to  the 
distribution  of  the  contractor's  costs;  amount 
equal  to  imputed  interest  costs  recorded  on 
an  annual  basis  over  the  lease  term 

Lease-purchase  with 
Substantial  Private 
Risk 

Amount  equal  to  asset  cost  recorded 
up-front;  amount  equal  to  imputed 
interest  costs  recorded  on  an  annual 
basis  over  the  lease  term 

Scored  over  lease  term  in  an  amount  equal 
to  the  annual  lease  payments 

Capital  Lease 

Amount  equal  to  asset  cost  recorded 
up-front;  amount  equal  to  imputed 
interest  costs  recorded  on  an  annual 
basis  over  the  lease  term 

Scored  over  lease  term  in  an  amount  equal 
to  the  annual  lease  payments 

Operating  Lease 

Amount  equal  to  total  payments  under 
the  full  term  of  the  lease  or  amount 
sufficient  to  cover  first-year  lease 
payments  plus  cancellation  costs 
recorded  up-front 

Scored  over  lease  term  in  an  amount  equal 
to  the  annual  lease  payments 

(CBO,  2003,  p.  9) 

Table  1.  The  Budgetary  Treatment  of  Leases  and  Private/Public  Ventures 


For  lease-purchases  and  capital  leases,  budget  authority  will  be  scored  against 
legislation  in  the  year  in  which  the  budget  authority  is  first  made  available.  The  recorded 
amount  is  the  estimated  net  present  value  of  the  Government’s  total  estimated  legal  obligations 
over  the  life  of  the  lease  term.  From  a  budget  perspective,  purchases,  lease-purchases,  and 
capital  leases  all  attempt  to  acquire  an  asset  over  its  total  life  and  are  scored  similarly.  The  only 
major  difference  involves  the  treatment  of  outlays  in  lease  purchases  with  substantial  private 
risk. 


Scoring  Policy  of  Operating  leases 

Operating  leases  are  different  from  capital  leases  or  lease  purchases  because  the 
lessee  has  no  intention  to  purchase  the  asset.  The  budget  authority  for  operating  leases  will  be 
scored  in  the  first  year  budget  authority  is  made  available  in  the  amount  sufficient  to  cover  the 
Government’s  legal  obligations  (OMB,  2006,  June).  Budget  authority  for  operating  leases  is 
scored  for  the  full  cost  of  future  lease  payments  in  the  first  year  of  a  lease;  or,  if  a  cancellation 
clause  exists,  budget  authority  for  the  first  year  is  scored  equal  to  the  first  year’s  payment  plus 
cancellation  fees,  with  following  years  to  be  scored  incrementally. 
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Advantages  of  Leases 

Leasing  provides  a  number  of  important  advantages  in  addition  to  reducing  the  budget 
authority  assigned  to  a  project.  Leasing  generally  offers  a  higher  degree  of  flexibility  in 
operating  assets,  allowing  modification  of  assets  to  meet  changing  needs.  A  compliment  of 
services  typically  is  included  with  a  lease,  allowing  an  organization  to  draw  on  the  expertise  and 
resources  of  the  lessor.  Leasing  also  conserves  capital,  which  would  be  required  in  either  a 
down  payment  or  outright  purchase.  In  the  private  sector,  lease  payments  can  be  considered 
an  operating  expense  and,  thus,  offer  an  important  tax  advantage.  As  a  non-tax  paying  entity, 
the  government  does  not  have  this  advantage.  However,  leasing  affords  a  government  agency 
the  ability  to  spread  the  acquisition  costs  over  multiple  budgetary  periods,  which  will  more  likely 
correspond  with  the  useful  life  of  the  equipment.  Significant  up-front  costs  of  direct  purchasing 
may  pressure  an  agency  to  settle  for  lower-priced  equipment  that  fails  to  meet  the  requirements 


Alternative  Financing  Agreements:  Public-private  Partnerships 

In  August  2003,  the  Government  Accountability  Office  (GAO)  published  Alternative 
Approaches  to  Finance  Federal  Capital ,  which  examined  the  increased  usage  of  alternative 
financing  by  Federal  agencies.  The  GAO  identified  ten  alternative  financing  approaches  used  by 
Federal  agencies  to  fund  projects: 

1.  Incremental  funding, 

2.  Operating  leases, 

3.  Retained  fees, 

4.  Real  property  swaps, 

5.  Sale-leasebacks, 

6.  Lease-leasebacks, 

7.  Public-private  partnerships, 

8.  Out-leases, 

9.  Share-in-savings  contracts,  and 

10.  Debt  issuance. 

The  GAO  report  further  recognized  that  these  arrangements  would  be  beneficial  to 
agencies  in  that  they  would  be  able  to  acquire  capital  assets  without  first  having  to  secure 
sufficient  appropriations  to  cover  the  full  cost  of  the  asset  (GAO,  2003,  August).  Of  these 
financing  approaches,  public-private  partnerships  (PPPs)  have  the  greatest  potential  for  DoD 
procurement  of  military  equipment. 

The  scoring  of  public-private  partnerships  has  been  a  very  controversial  and  important 
budget  issue  to  those  seeking  to  utilize  private-sector  resources  in  government  projects. 
Because  no  two  public-private  partnerships  (PPPs)  are  arranged  exactly  the  same,  each  PPP 
must  be  carefully  examined  prior  to  any  scoring  determination.  Several  of  these  financing 
agreements  will  be  examined  in  the  included  Case  Studies.  The  major  debate  revolves  around 
the  determination  of  financial  obligation  and  risk  incurred  under  each  of  these  agreements. 
Because  of  the  complexity  and  individuality  of  many  of  these  arrangements,  there  is  usually  no 
precedent  to  guide  the  scoring  of  these  arrangements. 
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PPPs  can  be  used  by  the  government  to  affordably  take  advantage  of  an  underutilized 
asset,  benefiting  from  private-sector  expertise,  or  leverage  private-sector  financing  in  the  short¬ 
term  to  acquire  a  public  asset.  Leasing  may  only  be  small  part  of  the  PPP.  In  some  cases,  the 
government  may  benefit  from  the  revenue  a  leased  asset  generates  rather  than  benefit  from  the 
use  of  an  asset — serving  as  the  lessor  rather  than  the  lessee  (CBO,  2003,  p.  26). 

Unfortunately,  the  Budget  Committees,  OMB  and  CBO  are  typically  conservative  in  their  scoring 
of  these  arrangements  and  typically  do  not  discount  the  inherent  benefits  of  these  contracts 
from  the  overall  budget  authority  assigned  to  the  contract.  The  result  is  up-front  budget  authority 
scoring  for  the  project,  which  may  exclude  the  legislation  from  being  passed.  In  Case  Study 
Number  One,  various  alternative  financing  strategies  involving  governmental  housing  and 
buildings  will  be  examined  for  potential  applications  to  finance  military  capital  acquisitions.  In 
another  Case  Study,  share-in-savings  contracts  will  be  examined  utilizing  the  Energy  Savings 
Performance  Contracts  case.  Together,  these  case  studies  will  demonstrate  how  current 
scoring  guidelines  are  used  to  score  alternative  financing  arrangements  based  solely  on  the 
financial  obligation  without  sufficient  regard  to  the  program’s  benefits. 


Barriers  to  Alternative  Financing 

A  2003  report  written  by  the  Congressional  Budget  Office  addressed  the  government’s 
concern  involving  the  use  of  long-term  leasing  agreements;  in  particular,  their  ability  to: 

1 .  Reduce  the  budget’s  ability  to  fully  depict  the  Federal  Government’s  financial 
commitments. 

2.  Undermine  fiscal  policy  by  circumventing  controls  such  as  limits  on  deficits  and  caps  on 
Federal  Spending. 

3.  Allow  an  agency  to  avoid  facing  the  full  costs  of  purchasing  an  asset  at  the  time  it 
decides  to  buy  it,  thus  making  acquisitions  that  are  not  cost-effective  more  likely. 

4.  Raise  the  costs  of  some  investments  because  a  lease  purchase  is,  over  the  life  of  an 
asset,  inherently  more  costly  to  the  government  than  a  direct  purchase  (CBO,  2003,  p.  ix 
summary). 

Due  to  these  concerns,  there  is  a  large  incentive  to  capitalize  the  majority  of  lease 
agreements,  which  scores  the  lease  similar  to  a  direct  purchase.  Since  the  lease  involves 
payments  over  time,  there  is  an  inherent  interest  cost  disadvantage  when  delaying  the  payment 
of  an  asset.  The  scorekeepers  use  the  prime  rate  or  an  average  of  the  interest  rate  of 
marketable  Treasury  securities  as  their  standard  discount  rate.  Since  private  leasing  firms 
require  a  return  that  exceeds  the  prime  rate,  leasing  arrangements  incur  an  additional  cost:  the 
difference  between  the  prime  rate  and  the  negotiated  rate.  The  scoring  rules  also  assume  that 
parity  exists  between  public  and  private  firms  when  operating,  managing,  or  maintaining  an 
asset.  The  additional  services  and  expertise  of  the  private-sector  firm  are  not  incorporated  into 
the  current  scoring  guidelines. 


Scoring  Case  Studies 

Practical  Usage  of  Alternative  Financing 

Introduction 

As  previously  mentioned,  the  GAO  identified  10  capital  financing  approaches  in  use  by 
government  agencies  as  alternatives  to  the  conventional  full-funding  approach  (GAO,  2003, 
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August).  Five  of  these  10  approaches  were  selected  for  examination  on  the  basis  of  their 
potential  application  towards  funding  large  DoD  procurements  of  capital  equipment.  These 
financing  strategies  include:  Incremental  funding,  Operating  leases,  Public-private  partnerships, 
Share-in-savings  contracts,  and  Debt  issuance.  The  combined  effects  of  these  are  reducing 
the  up-front  budgetary  impact  of  capital  projects,  making  use  of  all  existing  public  resources, 
and  incorporating  private-sector  efficiencies  within  government  projects,  which  has  been  a 
stated  goal  of  every  administration  since  Thomas  Jefferson’s. 

In  1996,  Congress  passed  legislation  in  the  National  Defense  Authorization  Act;  this  bill 
created  the  Military  Housing  Privatization  Initiative  (MHPI)  to  address  the  costly  challenge  of 
maintaining  adequate  housing  for  service  members.  Of  the  300,000  military  housing  units  in 
existence,  an  estimated  200,000  units  were  in  need  of  repair  at  an  approximate  cost  of  $16 
billion,  which  would  restore  the  units  to  acceptable  conditions  (DoD,  1999).  The  Department  of 
Veterans  Affairs  (VA)  also  possesses  legislative  authority  to  utilize  alternative  financing 
techniques.  To  alleviate  the  large  up-front  costs  of  their  projects,  these  agencies  selected 
various  PPPs  as  alternatives  to  conventional  funding.  Several  MHPI  and  VA  projects  are 
presented  below  to  discuss  the  scoring  determination  and  potential  broader  application  to 
capital  procurement  for  the  DoD. 

Public  Private  Partnerships  (PPPs) 

In  a  2003  study  by  the  GAO,  PPPs  were  identified  as  the  most  prevalent  alternative 
financing  method,  with  over  54  different  agreements  in  existence  within  US  agencies  (GAO, 
2003,  August).  PPPs  are  a  particularly  popular  alternative-financing  technique  for  the  DoD  due 
to  their  great  flexibility  and  ability  to  apply  private-sector  capital  and  expertise  to  public  needs 
and  resources.  In  this  symbiotic  relationship,  each  party  benefits  from  its  participation  in  the 
partnership.  The  government  is  unable  to  be  the  most  efficient  provider  of  all  necessary 
services  and  equipment  items  for  the  public  sector.  OMB  Circular  A-76  acknowledges  this 
reality  and  provides  guidelines  with  which  to  outsource  public  requirements  to  the  private  sector 
and  promote  efficiency  (OMB,  2003,  May).  In  some  cases,  adaptable  technologies  or  industrial 
capacity  already  exist  in  the  private  sector  that  could  address  the  requirements  of  the  military.  A 
PPP  can  be  formed  to  exploit  these  opportunities  in  a  manner  conventional  full-funding 
procurement  cannot. 

Despite  the  efficiencies  of  PPPs,  the  scoring  of  PPP  legislation  has  become  increasingly 
conservative — limiting  the  flexibility  originally  granted  by  statuary  authority  to  several  Federal 
agencies.  The  CBO  and  OMB  believe  that  Federal  agencies  are  using  special  purpose  public- 
private  ventures  as  a  way  to  access  private  capital  without  triggering  lease-purchase  guidelines 
and  to  avoid  recording  obligations  up-front  in  their  budgets.  This  section  will  discuss  these 
concerns  and  other  scoring  issues  using  several  examples  from  the  DoD’s  privatization  of 
military  housing  and  the  VA’s  enhanced-use  lease  authority. 

The  majority  of  PPPs  involve  the  Federal  Government’s  real  property  or  other 
underutilized  assets  that  can  be  developed,  revitalized,  or  managed  by  the  private  sector.  The 
key  element  of  a  PPP  is  that  the  government  possesses  some  non-monetary  asset  that  has 
value  to  the  private  sector.  In  a  typical  fully  funded  contract,  the  government  must  set  aside 
funds  sufficient  to  cover  all  obligations  in  the  first  year  of  the  project.  In  PPP  agreements,  the 
government  is  able  to  barter  an  asset  or  use  existing  conditions  in  lieu  of  full  payment  to  reduce 
their  obligations.  These  assets  can  include  loan  guarantees,  longer  lease  terms,  debt  issuance, 
guaranteed  minimum  rates  of  occupancy,  or  even  the  transfer  of  the  asset  at  the  completion  of 
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the  lease  term.  Figure  1  depicts  the  wide  degree  of  versatility  of  PPP  contracts  in  managing 
responsibility  throughout  the  life  of  an  asset. 
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Figure  1.  Degree  of  Government  Responsibility  in  PPP  Contracts 

(Dovey  &  Eggers,  2007,  p.  5) 

Below  is  a  listing  of  the  most  common  PPP  relationships  in  existence. 

Design-Build  (DB):  Under  this  model,  the  government  contracts  with  a  private  partner 
to  design  and  build  a  facility  in  accordance  with  the  requirements  set  by  the  government.  After 
completing  the  facility,  the  government  assumes  responsibility  for  operating  and  maintaining  the 
facility.  This  method  of  procurement  is  also  referred  to  as  Build-Transfer  (BT). 

Design-Build-Maintain  (DBM):  This  model  is  similar  to  Design-Build  except  that  the 
private  sector  also  maintains  the  facility.  The  public  sector  retains  responsibility  for  operations. 

Design-Build-Operate  (DBO):  Under  this  model,  the  private  sector  designs  and  builds  a 
facility.  Once  the  facility  is  completed,  the  title  for  the  new  facility  is  transferred  to  the  public 
sector,  while  the  private  sector  operates  the  facility  for  a  specified  period.  This  procurement 
model  is  also  referred  to  as  Build-Transfer-Operate  (BTO). 

Design-Build-Operate-Maintain  (DBOM):  This  model  combines  the  responsibilities  of 
design-build  procurements  with  the  operations  and  maintenance  of  a  facility  for  a  specified 
period  by  a  private-sector  partner.  At  the  end  of  that  period,  the  operation  of  the  facility  is 
transferred  back  to  the  public  sector.  This  method  of  procurement  is  also  referred  to  as  Build- 
Operate-Transfer  (BOT). 

Build-Own-Operate-Transfer  (BOOT):  The  government  grants  a  franchise  to  a  private 
partner  to  finance,  design,  build  and  operate  a  facility  for  a  specific  period  of  time.  Ownership  of 
the  facility  is  transferred  back  to  the  public  sector  at  the  end  of  that  period. 

Build-Own-Operate  (BOO):  The  government  grants  the  right  to  finance,  design,  build, 
operate  and  maintain  a  project  to  a  private  entity,  which  retains  ownership  of  the  project.  The 
private  entity  is  not  required  to  transfer  the  facility  back  to  the  government. 

Design-Build-Finance-Operate/Maintain  (DBFO,  DBFM  or  DBFO/M):  Under  this 
model,  the  private  sector  designs,  builds,  finances,  operates  and/or  maintains  a  new  facility 
under  a  long-term  lease.  At  the  end  of  the  lease  term,  the  facility  is  transferred  to  the  public 
sector.  In  some  countries,  DBFO/M  covers  both  BOO  and  BOOT. 
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PPPs  can  also  be  used  for  existing  services  and  facilities  in  addition  to  new  ones.  Some 
of  these  models  are  described  below. 

Service  Contract:  The  government  contracts  with  a  private  entity  to  provide  services 
the  government  previously  performed. 

Management  Contract:  A  management  contract  differs  from  a  service  contract  in  that 
the  private  entity  is  responsible  for  all  aspects  of  operations  and  maintenance  of  the  facility 
under  contract. 

Lease:  The  government  grants  a  private  entity  a  leasehold  interest  in  an  asset.  The 
private  partner  operates  and  maintains  the  asset  in  accordance  with  the  terms  of  the  lease. 

Concession:  The  government  grants  a  private  entity  exclusive  rights  to  provide,  operate 
and  maintain  an  asset  over  a  long  period  of  time  in  accordance  with  performance  requirements 
set  forth  by  the  government.  The  public  sector  retains  ownership  of  the  original  asset,  while  the 
private  operator  retains  ownership  over  any  improvements  made  during  the  concession  period. 

Divestiture:  The  government  transfers  an  asset,  either  in  part  or  in  full,  to  the  private 
sector.  Generally,  the  government  will  include  certain  conditions  with  the  sale  of  the  asset  to 
ensure  that  improvements  are  made  and  citizens  continue  to  be  served  (Dovey  &  Eggers,  2007, 
p.  5). 


PPP  Examples:  Government  Privatization  Initiative 

The  statuary  authority  originally  granted  in  1996,  and  later  made  permanent  in  2005, 
allows  the  government  to  enter  into  public-private  partnerships  without  individual  project 
approval  from  Congress  (10  USC  2871-2885).  The  relative  complexity  of  PPP  contracts 
frequently  generates  disagreements  amongst  the  CBO,  OMB,  and  agency  representatives 
concerning  the  interpretation  of  the  scoring  guidelines.  The  goal  of  the  CBO  and  OMB  is  to 
provide  to  decision-makers  an  accurate  account  of  the  amount  of  legal  obligations  of  the  federal 
government.  PPPs  represent  a  valuable  method  of  accessing  private  capital  and  expertise 
independent  of  the  scoring  determination  made  by  the  CBO/OMB.  The  following  DoD  and  VA 
case  studies  analyze  the  value  of  the  PPPs  and  the  scoring  issues  in  the  cases. 

Public-private  Partnership  Case  Study  1:  Ft.  Hood  Family  Housing,  LLP 

Description  of  Project 

In  2001 ,  Fort  Hood  Family  Housing  was  selected  as  one  of  the  first  PPPs  by  the  Army 
when  it  contracted  Actus  Lend  Lease  to  manage  all  aspects  of  the  development,  financing, 
construction,  and  property  management  for  the  Fort  Hood  Family  Housing  project  (Fort  Hood 
Family  Housing,  2007).  The  partnership  detailed  a  50-year  lease  to  maintain  the  5,912  units 
located  at  Fort  Hood,  Texas  (CBO,  2003,  p.  28). 

Financial  Details 

At  the  conclusion  of  the  initial  50-year  lease,  the  Army  has  the  option  to  renew  for 
another  25-year  lease  term.  If  the  Army  does  not  renew,  all  assets  remain  government 
property. 
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The  housing  project  has  an  estimated  cost  of  $260  million.  The  burden  of  that  cost  will 
be  divided:  $186  million  will  come  in  the  form  of  a  loan  entered  into  by  the  partnership;  Bank 
One  will  provide  $20  million  in  private  equity;  and  the  Army  will  invest  $52  million  in  equity. 

Actus  will  also  provide  $6  million  in  equity  at  the  end  of  the  fifth  year  for  additional 
development.  The  contract  also  provides  Actus  with  a  preferred  return  on  equity  of  10-12%, 
and  a  portion  of  partnership  earnings  up  to  a  predetermined  ceiling.  Actus  will  also  receive 
payment  equal  to  a  fixed  percentage  of  the  project’s  gross  revenue  for  its  management 
services. 

Scoring  Impact  and  Issues 

The  Army  was  able  to  obtain  in  excess  of  $273  million  in  financing  for  an  up-front  cost  of 
$52  million  (CBO,  2003,  p.  42).  Only  the  Army’s  direct  investment  of  $52  million  was  scored  by 
the  OMB  as  an  immediate  obligation.  The  transfer  of  land  and  pre-existing  housing  units  to 
Actus  had  no  budget  impact  based  on  the  absence  of  any  cash  transaction  between  the  two 
entities.  A  summary  of  budgetary  treatment  of  asset  sales  and  barters  by  the  CBO  and  OMB  is 
included  in  the  table  below.  The  rental  of  the  housing  units  to  service  members  was  viewed  as 
individual  transactions  between  private  parties.  This  distributed  the  budget  impact  for  the 
housing  expenditures  to  an  annual  expense  vice  an  up-front  cost.  Additionally,  the  $186  million 
obtained  via  loan  is  viewed  by  the  Army  as  debt  of  a  private  entity  and  not  the  government. 
According  to  the  contract  details,  the  Army  does  not  have  a  legal  obligation  to  cover  the  costs  of 
the  partnership’s  financing.  However,  the  housing  units  are  located  on  government  land,  and  the 
management  terms  of  the  contract  effectively  place  the  housing  under  government  control. 

The  actual  budgetary  impact  and  actual  cost  of  this  PPP  has  particular  significance  as 
the  Fort  Hood  Family  Housing,  LP,  was  one  of  the  first  PPPs  initiated  by  the  DoD.  The  scoring 
debate  has  two  clearly  polarized  sides.  From  the  scorekeepers’  perspective,  the  government’s 
total  obligations  remain  hidden  in  the  financial  framework  of  the  partnership,  blinding  Congress 
of  the  needed  information  to  calculate  future  budget  impact.  Also,  an  important  underlying  issue 
remains:  is  this  type  of  partnership  actually  cost-effective?  The  deal  stipulates  a  mandated  10- 
12%  return  on  equity  plus  a  management  fee  based  on  the  partnership  revenues.  Could  the 
government  provide  this  service  at  similar  cost  and  service  levels?  Another  critical  issue  for  the 
CBO/OMB  is  the  long-term  (50  years)  lease  agreement  that  represents  a  long-term  commitment 
to  the  Fort  Hood,  Texas  base.  The  long-term  lease  limits  the  year-to-year  budget  control  of 
Congress  and  obligates  the  Federal  government  to  unspecified  future  obligations. 

From  the  service  and  partnership  perspective,  the  PPP  allows  the  DoD  to  immediately 
resolve  the  issue  of  substandard  and  insufficient  military  housing  that  threaten  the  quality  of  life 
and  retention  of  the  military.  The  costs  of  the  project  are  distributed  throughout  the  life  of  the 
project.  The  venture  utilizes  the  housing  allowances  of  the  individual  service  members  to 
finance  the  agreement  over  the  lease  term.  Actus  Lend  Lease,  with  over  30,000  managed 
units,  is  able  to  offer  considerable  private  expertise  that  helps  achieve  a  more  efficiently  run 
housing  project  and  higher  customer-satisfaction  levels  (Fort  Hood  Family  Housing,  2007).  The 
agreement  also  alleviates  the  DoD  of  housing  funding  that  can  now  support  other  missions  such 
as  Iraqi  reconstruction  and  development. 

PPPs  also  provide  stronger  incentives  to  complete  the  project  on-time  and  under  budget. 
In  2003,  the  United  Kingdom’s  National  Audit  Office  reported  that  73%  of  non-Private  Finance 
Initiative  (PFI)  projects  were  over  budget,  and  70%  were  delayed — versus  only  22%  of  PFI 
contracts  delivered  over  budget,  and  24%  delivered  late  (Dovey  &  Eggers,  2007,  p.  7).  The 


n.-.i'.uMiAftr.  v  hsiu,, 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  387- 


UK’s  previous  experience  in  public-private  partnerships  has  demonstrated  that  non-financial 
cost  factors  such  as  quality,  service,  timeliness,  and  expertise  can  often  justify  the  involvement 
of  the  private  sector  in  providing  public  financing. 

This  case  study  highlights  the  scoring  impact  of  these  alternative  financial  arrangements. 
For  the  Fort  Hood  Family  Housing  Project,  should  the  up-front  obligation  for  the  government  be 
scored  at  $52  million  or  $273  million?  The  CBO  asserts  that  although  only  a  small  portion  of  the 
total  investment  has  been  fronted  by  the  DoD,  the  DoD  has  overall  controlling  interest  in  the 
project.  The  venture  is  structured  to  fulfill  the  service  needs;  the  Army  shares  in  the  earnings  of 
the  venture  above  a  threshold,  and  also  controls  the  housing  units  at  the  end  of  the  lease. 
Additionally,  military  tenants  have  preferential  status  for  obtaining  occupancy,  and  the  venture 
must  maintain  affordable  rents  for  service  members  (CBO,  2003,  p.  29).  The  argument  is  made 
that  Fort  Hood  Family  Housing  Project  is  a  purely  government-driven  project. 

The  issue  is  not  whether  or  not  the  contract  is  structured  for  the  service’s  interests — of 
course  it  is.  The  issue  is  whether  this  type  of  alternative  financing  is  beneficial  to  both  the 
service  and,  more  importantly,  the  government  as  a  whole.  In  this  case,  the  Army  should  have 
the  flexibility  to  improve  existing  military  housing  with  a  lower  up-front  cost  of  $52  million. 
Congress  and  Army  leaders  must  realize,  however,  that  the  total  obligation  to  the  government 
will  exceed  the  $273  million  total  investment  upon  conclusion  of  the  project.  A  balance  between 
Congress’  desire  to  control  the  purse  versus  the  services’  need  to  supply  critical  services  to 
their  members  must  be  reached.  The  budgetary  impact  of  CBO/OMB  scoring  will  continue  to  be 
analyzed  in  the  following  case  studies. 

Public-private  Partnership  Case  Study  2:  Chicago  West  Side  Regional 

Headquarters  (CWSRH) 

The  Department  of  Veteran’s  Affairs  (VA)  was  granted  authority  to  enter  into  enhanced- 
use  lease  contracts  in  1991  (38  USC  8161-8169).  The  legislation  allows  the  VAto  lease 
government  land  to  private  entities  for  up  to  75  years  for  the  purpose  of  developing  the  land  for 
VA  or  private  needs.  Payments  resulting  from  the  lease  can  be  used  by  the  VA  without  further 
Congressional  oversight  (CBO,  2003,  p.  31 ).  The  VA  then  has  the  option  of  leasing  back  the 
privately  developed  facilities  for  their  uses.  The  VA  can  enter  into  these  agreements  without 
Congressional  approval  and  only  must  notify  Congress  within  60  days  of  the  enhanced-use 
lease  agreement.  Enhanced-use  leases  are  particularly  attractive  to  the  VA  due  to  their  vast 
holdings  of  underdeveloped  land  and  facilities. 

Description  of  Project 

The  Chicago  West  Side  Regional  Headquarters  (CWSRH)  project’s  enhanced-use  lease 
is  an  example  of  the  flexibility  of  PPPs,  but  the  project  also  presents  difficult  scoring  issues  to 
the  CBO/OMB.  In  2002,  the  VA  entered  into  a  series  of  agreements  used  to  fund  a  new  $60 
million  Chicago  headquarters  building  and  parking  facility  (CBO,  2003,  p.33). 

Financial  Details 

The  project  involved  numerous  interdependent  agreements.  West  Side  Enhanced-use 
Lease  Trust  was  created,  with  the  VA  named  as  sole  beneficiary.  A  four-acre  plot  adjacent  to 
the  VA  Medical  Center  in  downtown  Chicago  was  included  in  the  trust,  using  a  35-year 
enhanced  use  lease  agreement.  MedPark,  a  private  contractor,  would  be  responsible  for  the 
construction,  outfitting,  and  management  of  the  office  and  parking  facilities.  The  Illinois 
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Department  Finance  Authority  would  issue  $9  million  in  taxable  revenue  bonds  to  help  fund  the 
project.  The  proceeds  from  the  bond  issuance  would  be  loaned  to  the  Trust  to  pay  for  the 
design  and  construction  of  the  facility.  Under  the  lease  terms  for  the  building,  the  VA  is 
obligated  to  a  two-year  lease  for  a  minimum  of  95%  of  the  office  and  parking  facilities.  The 
leases  are  automatically  renewing  unless  the  VA  renders  written  notice  prior  to  the  end  of  the 
lease  term.  Additionally,  if  the  VA  occupies  any  portion  of  the  building,  it  must  cover  at  a 
minimum  the  amortization  and  interest  of  the  trust’s  loans  plus  all  the  trustee’s  expenses  (CBO, 
2003,  p.  34). 

Scoring  Impact  and  Issues 

In  October  2001,  the  OMB  and  VA  settled  how  obligations  and  outlays  would  be  treated 
for  lease-back  agreements  (CBO,  2003,  p.  44).  A  leaseback  agreement  is  a  lease  in  which  the 
government  is  the  lessor  vice  the  lessee.  The  agreement  stipulated  that  VA  leasebacks  of 
terms  up  to  two  years  in  length  would  be  treated  as  operating  leases,  as  long  as  the  VA  had  no 
right  of  first  refusal  on  future  lease  terms.  The  property  lease  was  drafted  to  be  a  non-cash 
barter  transaction  without  budget  impact.  The  revenues  received  by  the  VA  from  the  trust  would 
offset  the  VA’s  initial  investment  and  be  under  the  agency’s  discretion.  The  initial  lease  of  the 
building  was  designed  to  be  an  operating  lease,  and  the  borrowing  of  the  Trust  to  be  private 
borrowing.  The  VA  does  not  have  right  of  first  refusal  for  future  leases  as  mandated  by  their 
2001  agreement  with  the  OMB.  The  CBO  is  concerned  with  the  VA’s  obligation  to  cover  the 
cost  of  capital  for  the  T rust  even  if  the  agency  reduces  its  usage  of  the  facility. 

The  CBO  points  out  that  the  CWSRH  enhanced-use  lease  agreement  represents  a 
significant  long-term  obligation  by  the  VA  and  is  not  actually  limited  to  the  initial  two-year  lease 
term.  As  such,  the  budget  impact  of  the  project  far  exceeds  the  VA’s  estimation.  Congressional 
scoring  is  only  rendered  on  new  legislation;  since  the  project  was  passed  under  existing 
authority,  the  scoring  issues  remain  unresolved.  From  the  VA’s  perspective,  the  agreement  was 
crafted  with  only  limited,  future  risk  to  the  government.  The  facilities  were  built  in  a  highly 
popular  section  of  downtown  Chicago — increasing  the  likelihood  of  finding  replacement  tenants 
if  demand  for  usage  fell  below  initial  levels.  Additionally,  the  VA  benefited,  as  sole  beneficiary  of 
the  Trust,  from  proceeds  from  the  leasing.  The  obligation  of  the  Trust  to  the  VA  would  help  the 
Trust  obtain  funding  and  reduce  risk  from  private  creditors. 

The  VA’s  Chicago  project  is  a  demonstration  of  how  a  government  agency  was  able  to 
utilize  an  underdeveloped  asset  to  fulfill  an  immediate  need.  The  project  was  designed  to  limit 
the  initial  up-front  cost  of  the  venture  to  the  initial  two-year  lease  agreement,  with  construction 
and  design  of  the  facilities  to  be  paid  for  via  private  equity.  Future  lease  agreements  would  be 
entirely  governed  by  the  private  trust  for  the  35-year  agreement,  with  the  VA  receiving 
preferential  treatment  in  facility  usage.  Utilizing  the  legislative  authority,  the  VA  was  able  to 
construct  the  optimal  size  facility  and  benefit  from  any  private  usage  of  the  facility.  The  project 
is  an  example  of  how  a  PPP  was  used  to  finance  and  successfully  create  an  otherwise 
unaffordable  project. 

An  obstacle  to  greater  usage  of  this  type  of  funding  is  the  budgetary  treatment  from  the 
OMB  and,  particularly,  the  CBO.  In  2002,  HR  3947,  the  Federal  Property  Management  Reform 
Act,  was  introduced  that  would  give  federal  landholding  agencies  additional  authorities  in 
acquiring,  managing,  improving,  and  disposing  of  their  property  assets;  it  also  provided 
incentives  to  manage  these  assets  efficiently  (CBO,  2002).  Although  the  bill  did  not  receive  the 
necessary  approval,  it  did  clarify  the  position  of  the  CBO  towards  PPPs.  In  its  Cost  Estimate  for 
the  Bill,  the  CBO  stated  it  viewed,  “hybrid  entities  like  public-private  business  ventures”  as 
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governmental.  Meaning  that,  since  the  purpose  of  the  venture  is  mostly  or  entirely 
governmental,  any  borrowing  or  outside  financing  activities  would  be  viewed  as  new  federal 
borrowing  authority.  Additionally,  it  felt  that  most,  if  not  all,  of  the  public-private  ventures  should 
be  subject  to  the  lease-purchase  scoring  guidelines  contained  in  OMB  Circular  A-1 1 . 

The  scoring  of  private  involvement  remains  a  frustrating  issue  as  the  CBO  reserves  the 
right  to  alter  its  interpretation  of  the  scoring  guidelines.  For  six  years  (1 996-2002),  the  CBO 
scored  military  housing  ventures  consistently  with  the  OMB.  However,  in  2002,  the  CBO 
changed  its  position,  viewing  the  ventures  as  additional  borrowing.  In  regards  to  share-in- 
savings  contracts,  the  CBO  reversed  a  decade-old  policy  of  scoring  ESPCs  budget-neutral  to 
scoring  them  as  additional  budget  authority.  Other  agencies,  such  as  General  Services 
Administration  (GSA),  support  legislation  that  expands  the  authority  to  utilize  private 
partnerships  (Perry,  2002,  April  18).  While  the  CBO’s  role  is  to  remain  objective  and  impartial, 
its  interpretation  of  scoring  guidelines  dictates  policy  for  privatization  initiatives.  If  Congress 
seeks  to  build  on  the  recent  successes  in  military  housing  or  VA’s  enhanced-use  contracts,  its 
members  should  offer  directed  scorekeeping  that  promotes  efficient  economic  use  of  DoD 
resources.  It  is  our  belief  that  the  efficiencies  of  these  ventures  can  be  translated  on  a  larger 
scale  to  the  procurement,  management,  and  disposal  of  military  capital  equipment.  The  DoD 
can  more  efficiently  procure  and  manage  its  assets,  but  only  if  it  receives  legislative  authority 
and  budgetary  treatment  allowing  it  to  do  so. 

The  decision  to  undertake  a  project  must  be  separated  into  two  parts: 

1 .  Is  the  project  worthwhile  to  undertake? 

a.  Do  the  benefits  exceed  costs? 

b.  Does  the  return  exceed  the  required  rate  of  return  on  investment? 

c.  Does  this  project  warrant  the  limited  resources  that  it  will  consume? 

2.  Given  that  this  project  is  worthwhile,  what  is  the  best  method  to  finance  the  project? 

ESPC  Case  Study 

Energy  Savings  Performance  Contracting  (ESPC)  is  the  most  cost-effective  means  of 
completing  building  energy  upgrades  and  associated  savings.  The  concept  has  existed  since 
1992,  but  it  was  not  implemented  by  the  Department  of  Energy  until  1995  (DoE,  2006,  June). 
ESPC  is  a  means  of  using  utility  savings  to  pay  for  all  project  costs.  There  are  many 
possibilities  of  projects,  such  as  energy-management  systems,  interior  and  exterior  lighting, 
boiler  replacement  or  repair  of  steam  systems,  and  replacement  of  Heating,  Ventilation,  and  Air 
Conditioning  (HVAC)  (Washington  State  Department  of  General  Administration,  2007).  This 
form  of  contracting  normally  guarantees  project  costs,  savings  and  performance  of  installed 
equipment.  However,  the  majority  of  risk  is  borne  by  the  contractor,  not  the  government.  The 
government  must  fully  fund  the  project — which  often  causes  debate  about  the  direct  costs  and 
overall  benefit. 

The  Department  of  Energy  explains: 

An  ESPC  project  is  a  partnership  between  the  customer  and  an  energy  service  company 
(ESCO).  The  ESCO  conducts  a  comprehensive  energy  audit  and  identifies 
improvements  that  will  save  energy  at  the  facility.  In  consultation  with  the  agency 
customer,  The  ESCO  designs  and  constructs  a  project  that  meets  the  agency’s  needs 
and  arranges  financing  to  pay  for  it.  The  ESCO  guarantees  savings  sufficient  to  pay  for 
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the  project  over  the  term  of  the  contract.  After  the  contract  ends,  all  additional  cost 
savings  accrue  to  the  agency.  Contract  terms  up  to  25  years  are  allowed.  (DoE,  2006, 
June) 

Since  2005,  more  than  400  federal  ESPC  projects,  in  46  different  states,  by  19  different 
federal  agencies  (altogether  worth  $1.9  billion)  have  generated  $5.2  billion  in  energy  cost 
savings  (2006,  June). 

The  use  of  ESPCs  is  ideal  for  organizations  which  seek  out  alternative  means  of  funding 
programs.  As  the  Department  of  Defense  (DoD)’s  discretionary  portions  of  the  budget  continue 
to  become  strained,  high  competition  for  those  funds  may  leave  critical  programs  dry.  Many 
facilities  throughout  the  DoD  were  built  shortly  after  World  War  II.  Few  new  facilities  have  been 
built  replacing  the  old.  Dated  DoD  equipment  and  assets — such  as  the  B-52  bomber,  SH-60 
helicopter  and  many  others — are  continuously  being  tunneled  additional  funds.  This  funding  is 
higher  than  normal  funding  for  these  assets  due  to  increased  maintenance,  poor  fuel  economy, 
dated  insulation  techniques,  and  lack  of  funding  to  support  replacements.  Thus,  the  DoD 
continues  to  live  with  existing  problems.  The  ESPC  is  a  means  to  cut  costs  while  continuing 
overall  functionality  of  facilities  and  assets.  Other  means  of  financing,  such  as  PPPs  and 
various  forms  of  leases,  are  used  successfully  today  by  the  private  sector  and  will  become  a 
way  of  life  for  many  organizations. 

A  perfect  example  of  the  benefits  of  an  ESPC  pertains  to  many  homeowners.  A 
homeowner  will  evaluate  the  cost  of  improving  his/her  home  with  the  expected  benefits.  The 
homeowner  may  attempt  to  determine  some  form  of  payback  period  or  return  on  investment. 
The  homeowner  must  determine  the  means  of  financing  such  improvements  as  well.  The 
government  and  its  facilities  are  no  different.  But  many  vendors  are  willing  to  offer  their 
supplies  and  equipment  to  help  defer  the  required  payments  over  some  time  frame,  but  at  some 
higher  price.  Assume  the  proposed  cost  to  renovate  or  improve  a  home  was  $10,000.  This 
improvement  would  replace  the  windows,  lighting  and  appliances.  The  home  would  become 
more  efficient  and  reduce  utility  costs.  The  vendor  and  homeowner  would  agree  upon  some 
baseline  on  expenses  once  improvements  were  installed,  and  the  difference  would  be  used  to 
“pay  off’  the  vendor  for  its  services.  If  there  are  no  savings,  the  vendor  does  not  get  paid. 
Assume  for  contract  period  is  eight  years.  Table  2  illustrates  two  scenarios.  The  first  assumes 
the  homeowner  paid  the  vendors  $10,000  up-front;  then  he  realizes  a  30%  or  50%  reduction  in 
his  existing  $5,000  annual  utility  expense.  The  second  scenario  assumes  the  homeowner  pays 
for  the  improvement  in  some  agreed-upon  ESPC  with  the  vendors  over  five-years — with  the 
same  50%  reduction  in  annual  utility  expense  and  a  3.00%  rate  of  inflation. 
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0 

Payback  period  for  scenario  1  would  be  $10,000/$!, 500  which  is  6.67  years. 

There  is  no  payback  period  for  scenario  2  due  to  not  making  any  investment  into  the  project. 

Table  2.  ESPC  Scenarios 


Table  2  illustrates  in  Scenario  1  that  if  the  improvement  is  fully  funded  up-front  in  Year  0, 
there  is  a  positive  Net  Present  Value  on  the  investment  for  both  the  30%  and  50%  reductions  in 
utility  expenses.  Scenario  2  demonstrates  with  no  initial  investment,  a  positive  Net  Present 
Value  on  the  investment  for  both  the  30%  and  50%  reductions  in  utility  expenses.  The  vendor 
would  also  benefit  from  assisting  the  homeowner  with  the  improvements.  It  is  a  win-win 
situation.  The  homeowner  does  not  have  to  “fully  fund”  the  project  and  achieves  the  same 
result  with  an  alternate  form  of  financing.  Today,  many  private  companies  and  local 
governments  implement  ESPCs. 

Within  the  government  and  many  federal  agencies,  there  are  different  points  of  view 
pertaining  to  ESPCs  and  their  application  in  the  budget.  The  first  is  that  ESPCs  should  be 
scored  at  zero  because  they  pay  for  themselves.  The  other  is  that  the  funds  must  be  obligated 
in  case  of  contract  issues  such  as  The  Anti-deficiency  Act.  The  Anti-deficiency  Act,  also 
known  as  31  USC  Section  1301(a),  is  one  of  the  major  laws  in  which  the  Congress  exercises  its 
constitutional  control  of  the  public  purse.  Thus,  ESPCs  continue  to  be  debated,  and  their  role 
within  the  budget  is  uncertain.  Yet,  as  demonstrated  above,  ESPCs  are  clearly  a  viable  solution 
to  cut  costs  for  the  DoD’s  facilities  and  assets. 

The  Energy  Policy  Act  of  2005  set  new  federal  energy  goals.  The  act  states  that  one 
goal  was  to  “cut  energy  usage  by  2  percent  per  year  from  2006-201 5”  (DoE,  2007).  The  federal 
government  may  not  achieve  this  goal  without  improving  existing  conditions  at  its  facilities  or 
with  its  equipment.  The  budget  continues  to  be  strained  due  to  the  Global  War  on  Terrorism, 
increased  health  costs,  a  need  for  social  security  reform,  and  other  political  issues.  One  means 
to  cut  existing  costs  without  having  an  effect  on  the  budget  is  the  usage  of  ESPCs  for  existing 
facilities  and  equipment.  Recently,  a  step  in  the  right  direction  was  made  by  the  Air  Force.  On 
March  1,  2007,  the  Air  Force  awarded  Northrop  Grumman  a  contract  with  the  same  principles 
as  the  ESPC.  The  contract  was  for  the  initial  $12.5  million  E-8C  Joint  Surveillance  Target 
Attack  Radar  System  (STARS)  engine  replacement.  The  contract  defines  the,  “non-recurring 
engineering  work  required  to  replace  engines  on  the  E-8C  Joint  STARS  aircraft”  (Stratford, 
2007).  Other  enhancements  for  the  Boeing  707-based  platform  are  scheduled  with  a  similar 
contract.  The  Air  Force  decision  was  based  on  the  Boeing  700-300C  series  aircraft  refurbished 
by  Northrop  Grumman,  which  have  performed  much  better  than  the  Air  Force  E-8C.  Thus,  this 
has  created  an  opportunity  for  the  Air  Force  to  maximize  the  benefits  of  the  ESPC  to  improve 
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the  E-8C’s  reliability,  reduce  maintenance  costs,  and  make  the  aircraft  more  fuel-efficient — 
allowing  less  “in-air”  refueling  and  allowing  increased  on-station  time.  Thus,  both  the  contractor 
and  the  government  benefit  from  the  ESPC. 

New  technologies,  especially  energy-saving  advancements,  should  be  used  when 
feasible  in  a  world  where  prices  continue  to  rise  as  resources  continue  to  become  scarce. 

United  States  companies  should  be  provided  an  incentive  to  explore  new  technologies  and 
processes  to  innovate  and  create  savings  which  not  only  help  them,  but  improve  the  efficiency 
of  the  government  and,  thus,  benefit  the  taxpayer.  If  such  were  the  case,  ultimately  the 
productivity  of  our  country  would  increase,  resources  would  be  conserved,  and  the  economy 
would  grow  while  helping  fund  our  government. 

Conclusions  and  Recommendations 

Recommendations 

This  research  addressed  only  a  limited  number  of  the  total  available  options  to  finance  a 
capital  asset.  All  factors  being  equal,  the  time-value  of  money  dictates  that  payment  of  an  asset 
up-front  will  necessarily  be  less  costly  than  delayed  payment.  However,  the  analysis  presented 
here  has  hopefully  addressed  several  scenarios  in  which  government  agencies  would  be  able  to 
leverage  their  available  resources  and  incorporate  the  private  sector  via  some  form  of  Public- 
private  Partnership  to  achieve  a  cost-effective  alternative  to  up-front  funding.  The  current 
scoring  guidelines  in  OMB  Circular  A-1 1  provide  a  negative  bias  towards  using  alternative 
approaches  to  full-funding.  In  many  cases,  agencies  are  forced  to  seek  alternative  funding 
measures  or  do  without  the  asset.  Several  recommendations  to  modify  the  current  budget¬ 
scoring  process  and  scoring  guidelines  in  an  attempt  to  promote  improved  economic  efficiency 
in  public  projects  are  presented  below. 


Scoring  of  Leasing 

The  crucial  question  in  categorizing  a  lease  is  determining  what  constitutes  purchase  of 
an  asset.  Long-term  leases  that  provide  the  government  with  ownership  of  the  asset  are  scored 
up-front  in  an  amount  equal  to  the  net  present  value  of  the  future  lease  payments  for  the  asset. 
Conversely,  leases  that  provide  the  government  with  only  partial  use  of  the  asset’s  economic  life 
can  be  scored  in  annual  obligations  as  an  operating  lease  (CBO,  2003,  p.viii).  The 
scorekeepers  apply  strict  criteria  in  determining  between  an  operating  or  capital  lease.  The 
result,  and  intent  of  the  guidelines,  is  that  almost  all  DoD  equipment  is  acquired  via  purchase  or 
capital  leases.  The  OMB  guidelines  for  operating  leases  have  two  additional  requirements  to 
the  four  basic  criteria  used  by  the  Federal  Accounting  Standards  Board  (FASB).  These  two 
requirements  include: 

1 .  There  is  a  private-sector  market  for  the  asset. 

2.  The  asset  is  a  general-purpose  asset  rather  than  being  for  a  special  purpose  of  the 
government  and  is  not  built  to  the  unique  specification  of  the  government  as  lessee 
(OMB,  2006,  June,  pp.  3-4). 

These  two  rules  are  specifically  designed  to  eliminate  operating  leases  as  a  financing 
option  for  the  procurement  of  military  equipment.  Table  3  outlines  the  criteria  for  Public  vs. 
Private-Operating-Lease  determination. 
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OMB  Requirements  to  be  Considered  an 
Operating  Lease  (Public) 

(OMB,  2006,  June,  pp.  3-4) 

Basic  Criteria  in  Lease  Determination 
(Private) 

(Lee,  2003,  pp.  10-11) 

Ownership  of  the  asset  remains  with  the  lessor 
during  the  term  of  the  lease  and  is  not 
transferred  to  the  Government  at  or  shortly  after 
the  end  of  the  lease  period. 

The  lease  transfers  ownership  of  the  property  to 
the  lessee  by  the  end  of  the  lease  term. 

The  lease  does  not  contain  a  bargain-price 
purchase  option. 

The  lease  contains  an  option  to  purchase  the 
leased  property  at  a  bargain  price. 

The  lease  term  does  not  exceed  75%  of  the 
estimated  economic  lifetime  of  the  asset. 

The  lease  term  is  equal  to  or  greater  than  75% 
of  the  estimated  economic  life  of  the  leased 
property. 

The  present  value  of  the  minimum  lease 
payments  over  the  life  of  the  lease  does  not 
exceed  90%  of  the  fair  market  value  of  the  asset 
at  the  inception  of  the  lease. 

The  present  value  of  rental  and  other  minimum 
lease  payments,  excluding  that  portion  of  the 
payments  representing  executory  costs,  equals 
or  exceeds  90%  of  the  fair  value  of  the  leased 
property. 

The  asset  is  a  general  purpose  asset  rather 
than  being  for  a  special  purpose  of  the 
Government  and  is  not  built  to  unique 
specification  for  the  Government  as  lessee. 

There  is  a  private-sector  market  for  the  asset. 

Table  3.  Public  vs.  Private  Operating  Lease  Determination 


The  stricter  guidelines  were  adopted  in  1991  in  response  to  the  frequent  use  of  lease- 
purchases  in  the  1980s  to  acquire  assets,  including  propositioned  ships  or  buildings.  The  CBO 
cited  four  major  concerns  of  the  increased  use  of  leasing  that  helped  inspire  the  new  guidelines. 
It  asserts  that  leasing  has  the  ability  to: 

1 .  Reduce  the  budget’s  ability  to  fully  depict  the  Federal  Government’s  financial 
commitments; 

2.  Undermine  fiscal  policy  by  circumventing  controls  such  as  limits  on  deficits  and  caps  on 
discretionary  spending; 

3.  Allow  an  agency  to  avoid  facing  the  full  costs  of  purchasing  an  asset  at  the  time  it 
decides  to  buy  it,  thus  making  acquisitions  that  are  not  cost-effective  more  likely;  and 

4.  Raise  the  costs  of  some  investments  because  a  lease-purchase  is,  over  the  life  of  an 
asset,  inherently  more  costly  to  the  government  than  a  direct  purchase  (CBO,  2003,  p. 
ix). 

We  propose  to  limit  the  guidelines  to  the  four  basic  criteria  accepted  in  the  private  sector 
with  one  additional  caveat.  A  fifth  guideline  would  include  a  proposal  that  highlighted  the 
following  issues: 

1 .  The  estimated  total  use  (years)  of  the  asset  by  the  government. 

2.  The  reason  as  to  why  operating  leasing  would  be  preferred  over  direct-purchase,  lease- 
purchase  or  other  type  of  financing. 
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3.  Explicitly  address  ownership  options  for  the  asset.  Also  discuss  the  probability  the  asset 
would  be  damaged  in  its  use  and  ultimately  be  required  to  be  purchased. 

4.  Salvage  value  for  the  asset  at  completion  of  the  lease — discussing  any  outside  markets 
for  the  asset  to  determine  potential  market  value. 


The  proposal  would  be  submitted  to  the  OMB,  CBO,  and  Congressional  Budget 
Committees  as  part  of  the  legislative  process.  If  these  new  guidelines  for  operating  leases  were 
adopted,  greater  flexibility  would  be  restored  to  the  DoD  in  financing  its  requirements.  The 
guidelines  would  not  hinder  Congress’s  ability  to  allocate  financial  resources  effectively. 


Scoring  of  Alternative  Financing 

Alternative  financing  consists  of  almost  any  financing  option  or  combination  of  options 
that  can  be  used  in  lieu  of  conventional  full-funding.  The  private  sector  has  metrics  such  as 
profit  or  stock  price  that  help  motivate  corporate  executives  in  their  selection  of  the  most 
beneficial  financing  method  for  their  company.  Without  these  incentives,  the  federal  budget 
process  remains  a  delicate  balance  between  agency  needs  and  Congressional  control  of  the 
purse.  Current  scoring  guidelines  are  designed  to  provide  the  decision-makers  in  Congress 
with  the  most  informative  representation  of  current  and  future  government  obligations.  The 
legislation  also  has  the  effect  of  biasing  full-funding  versus  other  forms  of  financing.  Yet,  in 
certain  situations,  the  needs  and  resources  of  the  government  can  be  combined  with  the 
capabilities  of  the  private  sector  to  form  a  partnership  that  is  beneficial  to  both  parties.  Public- 
private  Partnerships  represent  the  most  practical  financing  method  available  that  harnesses 
these  capabilities  and  addresses  the  needs  of  the  DoD. 

Unfortunately,  the  financial  details  of  Public-private  Partnerships  are  typically  unique  and 
involve  complex  financial  relationships,  causing  few  useful  precedents  to  be  available  to  help 
predict  the  scoring  outcome.  The  National  Council  for  Public  Private  Partnerships  (NCPPP) 
cites  one  of  the  major  impediments  in  the  budget  scoring  policy  to  be  the  ambiguity  surrounding 
the  current  scoring  guidelines  (2007).  The  OMB  scoring  rules  represent  policy  vice  actual  hard- 
fast  rules  and  are  intentionally  vague  to  allow  interpretation  by  the  CBO  or  OMB.  Reviewing  the 
scoring  determination  through  an  open  forum  between  concerned  policies  would  not  only  clarify 
the  intent  of  the  rules,  but  also  improve  adherence  to  the  scoring  rules.  The  Council  also 
asserts  that  scoring  confusion  could  further  be  eliminated  if  an  “Alternative  Financing” 
committee  was  formed  by  the  OMB  to  assist  agencies  that  seek  private-sector  involvement 
(2007).  The  committee  can  be  established  independently  from  the  OMB  to  eliminate  any 
conflicts  of  interests  or  questions  of  neutrality. 

Additionally,  the  scoring  process  would  be  improved  if  the  scoring  rules  placed  greater 
emphasis  on  economic  efficiency  rather  than  the  determination  of  outlays.  For  instance,  share- 
in-savings  contracts  that  have  outlays  resulting  only  from  the  net  savings  to  the  government 
should  be  scored  as  budget  neutral  or  have  some  other  discount  factor  that  reflects  the  financial 
benefits  of  the  deal.  Public-private  Partnerships  are  particularly  penalized  in  this  manner  as 
many  benefits  from  these  ventures  do  not  have  an  explicit  value  that  can  be  readily  estimated. 
The  inability  to  easily  or  accurately  estimate  these  benefits  causes  them  to  be  ignored  in  the 
scoring  process.  In  these  cases,  representatives  from  the  prospective  Alternative  Financing 
committee  could  provide  their  best  estimate  of  the  projected  savings  of  private  involvement — 
either  by  discounting  the  budget  authority  scored  for  the  project  or  by  including  this  dollar 
amount  independent  of  the  scoring  estimate.  In  either  manner,  Congress  would  be  informed  of 
the  benefits  of  the  alternative  financing.  The  current  scoring  rules  are  overly  conservative  and 
neglect  to  include  the  majority  of  the  benefits  of  PPP  in  scoring  budget  authority. 
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Conclusion 


The  consistency  and  transparency  in  the  budget  process  that  were  the  intent  of  the 
scoring  guidelines  also  have  the  affect  of  altering  the  feasibility  of  alternative  financing  ventures. 
Currently,  there  are  many  opportunities  to  improve  the  aging  infrastructure  and  reduce  the 
lifecycle  costs  of  a  project  through  greater  private-sector  involvement.  A  major  impediment  to 
realizing  this  goal  is  the  interpretation  of  the  scoring  guidelines  by  the  CBO  and  OMB  and  the 
absence  of  legislation  authorizing  such  private-sector  participation.  The  government  would 
benefit  from  either  a  revision  to  the  current  scoring  guidelines  or  a  more  comprehensive 
interpretation  of  the  current  scoring  rules.  We  assert  that  if  these  changes  are  implemented, 
then  the  soldiers  and  sailors  in  the  field  would  have  a  better  probability  of  being  provided  the 
equipment  necessary  to  complete  their  National  Security  mission  at  a  time  when  we  are  faced 
with  ever-shortening  supply  of  money  during  the  Global  War  on  Terrorism. 
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Panel  12  -  Considerations  in  Making  Logistics  Support  Choices  for 
Weapon  Systems 


Thursday, 
May  17,  2007 


9:30  a.m.  -  Chair: 
11:00  a.m. 


Reuben  S.  Pitts  III,  Head,  Warfare  Systems  Department,  NSWC  Dahlgren 
Division 


Discussant: 


David  Lamm,  Associate  Professor  emeritus,  Naval  Postgraduate  School 


Papers: 


Outsourcing  for  Optimal  Results:  Six  Ways  to  Structure  an  Evaluation  of 
Alternatives 

Francois  Melese,  Naval  Postgraduate  School 

Alternative  Strategies  for  Managing  MK48  Intermediate  Maintenance 
Activity 

William  Lucyshyn,  University  of  Maryland 


Chair:  Reuben  S.  Pitts  ill,  Head,  Warfare  Systems  Department,  NSWC  Dahlgren  Division 


Discussant:  David  Lamm,  Professor  Emeritus  from  the  Graduate  School  of  Business  and  Public  Policy 
(GSBPP),  served  at  NPS  as  both  a  military  and  civilian  professor  from  1978  through  his  retirement  in 
January  2004,  teaching  a  number  of  acquisition  and  contracting  courses,  as  well  as  advising  thesis  and 
MBA  project  students.  During  his  tenure,  he  served  as  the  Academic  Associate  for  the  Acquisition  & 
Contract  Management  (815)  MBA  Curriculum,  the  Systems  Acquisition  Management  (816)  MBA 
Curriculum,  the  Master  of  Science  in  Contract  Management  (835)  distance  learning  degree,  and  the 
Master  of  Science  in  Program  Management  (836)  distance  learning  degree.  He  created  the  latter  three 
programs.  He  also  created  the  International  Defense  Acquisition  Resources  Management  (IDARM) 
program,  which  brings  acquisition  education  in-country  to  over  20  allied  nations.  Further,  he  created  the 
Advanced  Acquisition  Program  (AAP),  a  distance  learning  certificate  program  for  the  civilian  acquisition 
workforce  throughout  the  country.  Finally,  in  collaboration  with  the  GSBPP  Acquisition  Chair,  he 
established  and  served  as  PI  for  the  Acquisition  Research  Program,  including  inauguration  of  an  annual 
Acquisition  Research  Symposium.  He  also  developed  the  Master  of  Science  in  Procurement  & 
Contracting  degree  program  at  St  Mary’s  College  in  Moraga,  CA,  and  served  as  a  Professor  in  both  the 
St  Mary’s  and  the  George  Washington  University’s  graduate  programs. 

He  has  researched  and  published  numerous  articles  and  has  written  an  acquisition  text  entitled  Contract 
Negotiation  Cases:  Government  and  Industry,  1993.  He  served  on  the  editorial  board  for  the  National 
Contract  Management  Journal  and  was  a  founding  member  of  the  editorial  board  for  the  Acquisition 
Review  Quarterly ,  now  known  as  the  Defense  Acquisition  Review  Journal.  He  served  as  the  NPS 
member  of  the  Defense  Acquisition  Research  Element  (DARE)  from  1983-1990. 
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Prior  to  NPS,  he  served  as  the  Supply  Officer  aboard  the  USS  Virgo  (AE-30)  and  the  USS  Hector  (AR-7). 
He  also  had  acquisition  tours  of  duty  at  the  Defense  Logistics  Agency  in  Contract  Administration  and  the 
Naval  Air  Systems  Command  where  he  was  the  Deputy  Director  of  the  Missile  Procurement  Division. 

He  holds  a  BA  from  the  University  of  Minnesota  and  a  MBA  and  DBA  both  from  The  George  Washington 
University.  He  is  a  Fellow  of  the  National  Contract  Management  Association  and  received  that 
association’s  Charles  A.  Dana  Distinguished  Service  Award  and  the  Blanche  Witte  Award  for  Contracting 
Excellence.  He  created  the  NCMA’s  Certified  Professional  Contracts  Manager  (CPCM)  Examination 
Board  and  served  as  its  Director  from  1975-1990.  He  is  the  1988  NPS  winner  of  the  RADM  John  J. 
Schieffelin  Award  for  Teaching  Excellence. 
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Outsourcing  for  Optimal  Results:  Six  Ways  to  Structure  an 
Evaluation  of  Alternatives 


Presenter:  Francois  Melese,  PhD,  joined  the  NPS  faculty  in  1987.  He  earned  his  undergraduate  degree 
in  Economics  at  UC  Berkeley,  his  Master’s  at  the  University  of  British  Columbia  in  Canada,  and  his  PhD 
at  the  Catholic  University  of  Louvain  in  Belgium.  After  five  years  as  a  faculty  member  in  the  Business 
School  at  Auburn  University,  Francois  joined  NPS  as  part  of  the  Defense  Resources  Management 
Institute  (DRMI).  In  his  time  at  NPS,  he  has  taught  public  budgeting  and  defense  management  in  over  two 
dozen  countries  and  has  published  over  50  articles  and  book  chapters  on  a  wide  variety  of  topics.  More 
recently,  at  the  request  of  the  State  Department  and  NATO  Headquarters,  he  has  represented  the  US  at 
NATO  Defense  meetings  in  Hungary,  the  Ukraine,  Germany  and  Armenia.  His  latest  article  (co-authored 
with  Jim  Blandin  and  Sean  O’Keefe)  appeared  in  the  International  Public  Management  Review.  The 
article  (available  at  www.ipmr.net)  is  entitled  “A  New  Management  Model  for  Government:  Integrating 
Activity-Based  Costing,  the  Balanced  Scorecard  and  Total  Quality  Management  with  the  spirit  of  the 
Planning,  Programming  and  Budgeting  System.” 

Francois  Melese,  PhD,  Professor  of  Economics 
Defense  Resources  Management  Institute 
School  of  International  Graduate  Studies 
Naval  Postgraduate  School 
Monterey,  CA  93943 
Phone:  (831)656-2009 
E-mail:  fmelese@nps.edu 


Abstract 

This  study  presents  six  ways  to  structure  an  evaluation  of  alternative  outsourcing 
opportunities.  According  to  the  Office  of  Management  and  Budget’s  (OMB)  Circular  A-76,  the 
decision  to  outsource  government  positions  or  functions  involves  a  comparison  of  “bids”  or 
“proposals” — Invitations  for  Bids  (IFB)  for  well-defined,  routine  commercial  activities;  Requests 
for  Proposals  (RFP)  for  more  complex,  more  difficult  to  define  activities.  The  paper  suggests 
replacing  the  conventional  decision  sciences  approach  that  currently  dominates  defense 
guidance,  with  a  more  intuitive  constrained  optimization  approach  borrowed  from  economist’s 
“characteristics  approach  to  demand  theory.”  One  of  the  key  insights  derived  from  the  economic 
approach  is  that  alternatives  are  generated  as  part  of  a  two-step  optimization  and  appear  as 
(response)  functions  and  not  points  in  Cost-Effectiveness  space.  One  important  implication  is 
that  what  have  previously  been  viewed  as  dominated  (inferior)  alternatives  may  prove  to  be 
superior  under  different  budget  scenarios.  The  study  discusses  concepts  of  intra-  and  inter¬ 
program  analysis,  cost  as  an  independent  variable,  expansion  paths,  “knees  of  the  curve,”  and 
opportunity  costs,  and  offers  an  intuitive  discussion  of  the  hazards  of  applying  cost/effectiveness 
ratios  to  rank  alternatives.  In  the  spirit  of  government  competitions,  anyone  that  discovers 
another  constrained  optimization  approach  to  structure  a  cost-effectiveness  analysis  of 
alternative  outsourcing  opportunities  will  receive  a  cash  award  of  $100  from  the  author. 
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Alternative  Strategies  for  Managing  MK48  Intermediate 
Maintenance  Activity 


Presenter:  William  Lucyshyn  is  the  Director  of  Research  and  a  Senior  Research  Scholar  at  the  Center 
for  Public  Policy  and  Private  Enterprise  in  the  School  of  Public  Affairs  at  the  University  of  Maryland.  In 
this  position,  he  conducts  research  on  the  public  policy  challenges  posed  by  the  increasing  role 
information  technologies  play  in  both  the  public  and  private  sectors. 

Previously,  Mr.  Lucyshyn  was  a  member  of  the  Senior  Executive  Service  and  served  as  a  program 
manager  and  the  principal  technical  advisor  to  the  Director,  Defense  Advanced  Research  Projects 
Agency  (DARPA),  on  the  identification,  selection,  research,  development,  and  prototype  production  of 
advanced  technology  projects.  Prior  to  this  appointment,  Mr.  Lucyshyn  completed  a  25-year  career  in  the 
US  Air  Force,  serving  in  various  special  operations  and  acquisition  positions. 

He  received  his  Bachelor  Degree  in  Engineering  Science  from  the  City  University  of  New  York  in  1971.  In 
1985,  he  earned  his  Master’s  Degree  in  Nuclear  Engineering  from  the  Air  Force  Institute  of  Technology. 
He  was  certified  Level  III,  as  an  Acquisition  Professional  in  Program  Management  in  1994. 

Expertise:  Market-based  Government,  Information  Security  Policy,  Supply  Chain 

2101  Van  Munching  Hall 
College  Park,  MD  20742 
Phone:  (301)405-8257 
E-mail:  Lucyshyn@umd.edu 


Abstract 

Since  1972,  Sailors  have  delivered  more  than  10,000  MK481  torpedoes  to  Pacific  Fleet 
submarines.  During  that  29-year  period,  the  IMA  achieved  numerous  awards  and  earned  a 
reputation  for  outstanding  torpedo  reliability.  Manning  shortfalls  in  the  Navy’s  torpedo-man 
rating  drove  the  decision  to  outsource  production  to  the  private  sector.  In  2001,  the  MK48 
torpedo  Intermediate  Maintenance  Activity  (IMA)  at  Pearl  Harbor  was  outsourced  to  a  contractor 
team  led  by  Raytheon.  All  but  two  of  the  181  military  billets  that  existed  at  the  IMA  were 
reallocated  to  other  critical  areas. 

Outsourcing  is  a  management  strategy  that  contracts  out  organizational  activities  to 
vendors  or  suppliers  who  specialize  in  these  activities  in  order  to  perform  them  more  efficiently. 
Outsourcing  or  “contracting  out”  still  requires  the  government  to  remain  fully  responsible  for  the 
provision  of  all  services  and  management  decisions.  If  implemented  correctly,  outsourcing  can 
be  used  to  introduce  competitive  pressure.  This  pressure  generally  incentivizes  performance 
improvements  and  cost  reductions. 

This  case  study  will  compare  the  operation  of  the  Pearl  Harbor  IMA  with  that  of  the  Navy 
Submarine  Torpedo  Facility  IMA,  Yorktown,  which  is  still  manned  primarily  by  active-duty 
sailors. 


1  The  Mark  48,  in  production  since  1972,  is  a  heavyweight  torpedo  still  carried  on  all  US  submarines  and 
designed  to  detonate  on  contact  or  in  proximity  to  a  target. 
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Panel  13  -  Considerations  for  Acquisition  Process  Improvements 


Thursday, 
May  17,  2007 


9:30  p.m.-  Chair: 
11:00  a.m. 


Dr.  Nancy  L.  Spruill,  Director,  Acquisition  Resources  and  Analysis,  Office 
of  the  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics) 

Papers: 

Review  of  Defense  Acquisition  Structures  &  Capabilities 

Paul  Alfieri  and  Mark  D.  Lumb,  Defense  Acquisition  University 

An  Integrated  Portfolio  Management  Approach  to  Weapon  System 
Investments  Could  Improve  DoD’s  Acquisition  Outcomes 

Katherine  Schinasi,  US  Government  Accountability  Office 

DoD  Contract  Termination  Liability:  An  Analysis  of  Special  Termination 
Cost  Clause  (STCC) 

Rene  Rendon  and  John  Mutty,  Naval  Postgraduate  School 


Chair:  Dr.  Nancy  L.  Spruill,  Director,  Acquisition  Resources  and  Analysis,  Office  of  the  Under  Secretary 
of  Defense  (Acquisition,  Technology  and  Logistics),  received  her  Bachelor  of  Science  degree  in 
Mathematics,  in  1971.  From  1 971  to  1 983,  she  held  a  variety  of  positions  with  the  Center  for  Naval 
Analyses,  including  Technical  Staff  Analyst,  Professional  Staff  Analyst  and  Project  Director.  She  earned 
her  Master  of  Arts  in  Mathematical  Statistics  in  1975  followed  by  her  Doctorate  in  1980. 

Dr.  Spruill  served  on  the  staff  of  the  Office  of  the  Secretary  of  Defense  from  1983  to  1993.  Initially,  she 
was  the  Senior  Planning,  Programming,  and  Budget  Analyst  in  the  Manpower,  Reserve  Affairs  and 
Logistics  Secretariat.  Later,  she  served  as  the  Director  for  Support  and  Liaison  for  the  Assistant  Secretary 
of  Defense  for  Force  Management  and  Personnel.  Then  she  served  as  the  Senior  Operations  Research 
Analyst  in  the  Office  of  the  Assistant  Secretary  of  Defense  for  Program  Analysis  and  Evaluation. 

In  1 993,  she  joined  the  staff  of  the  Defense  Mapping  Agency  (DMA),  serving  as  the  Chief  of  Programs 
and  Analysis  Division  for  the  DMA  Comptroller.  Her  role  included  oversight  of  the  Agency's  $800M 
program.  Subsequently,  she  served  as  Acting  Deputy  Comptroller  and  was  a  member  of  the  Reinvention 
Task  Force  for  the  Vice  President's  National  Performance  Review.  Her  reengineering  work  was 
implemented  and  resulted  in  a  mapping  organization  that  is  customer  focused  and  reduced  in  the 
management  layers  from  eleven  to  three. 

In  March  1 995,  she  was  selected  as  the  Deputy  Director  for  Acquisition  Resources  for  the  Under 
Secretary  of  Defense  for  Acquisition  and  Technology  (USD(AT&L)).  In  February  1999,  she  was  appointed 
Director,  Acquisition  Resources  &  Analysis  (ARA)  for  USD(AT&L).  In  this  capacity  she  is  responsible  for 
the  coordination  of  all  defense  acquisition  and  technology  planning,  programming,  and  budgeting  process 
activities,  as  well  as  funds  control,  Congressional  actions  in  the  authorization  and  appropriations 
processes,  and  special  analyses  for  the  Under  Secretary.  She  also  manages  the  studies  program  for 
OSD  and  oversees  USD(AT&L)'s  office  automation  system  and  manages  its  information  system  network. 
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Dr.  Spruill  has  been  a  member  of  the  Senior  Executive  Service  since  1995.  She  is  a  certified 
Acquisition  Professional  and  an  active  member  of  the  American  Statistical  Association.  Her  many  honors 
and  awards  include  the  Defense  Medal  for  Exceptional  Civilian  Service,  the  Defense  Medal  for 
Meritorious  Civilian  Service,  and  the  Hammer  Award.  She  has  contributed  papers  in  publications  of  the 
statistics  and  defense  analyses  communities  and  authored  articles  in  the  general  press  on  how  politicians 
use  -  and  abuse  -  statistics. 
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Review  of  Defense  Acquisition  Structures  &  Capabilities 


Presenter:  Paul  Alfieri,  Defense  Acquisition  University,  graduated  from  the  United  States  Naval 
Academy,  Annapolis,  MD,  in  June,  1969.  He  completed  Navy  flight  training  in  November,  1970,  and  was 
designated  a  naval  aviator.  Dr.  Alfieri  served  24  years  in  the  US  Navy,  in  which  he  was  a  helicopter  pilot 
and  an  Aeronautical  Engineering  Duty  Officer.  While  in  the  Navy,  he  earned  a  Master’s  degree  in 
Aeronautical  Engineering  from  the  Naval  Postgraduate  School  in  Monterey,  CA,  in  1977.  Dr.  Alfieri  spent 
8  years  at  the  Naval  Air  Systems  Command  and  became  an  acquisition  specialist  there.  At  NAVAIR,  Dr. 
Alfieri  was  a  Deputy  Program  Manager  (HARM  missile)  and  Program  Manager  (F404  engine).  Since 
leaving  the  Navy,  he  completed  studies  for  his  doctorate  in  Education  at  Virginia  Tech  in  Blacksburg,  VA, 
in  1 998.  For  the  last  1 6  years,  he  has  been  on  the  faculty  of  the  Defense  Acquisition  University,  where  he 
has  served  as  Department  Chair  of  the  Test  &  Evaluation  Department  and  is  currently  the  Director  of 
Research.  In  2002,  Dr.  Alfieri  graduated  from  the  Industrial  College  of  the  Armed  Forces  (ICAF)  in 
Washington,  DC,  with  a  Master’s  degree  in  National  Security  Affairs. 

Author:  Mark  D.  Lumb,  Defense  Acquisition  University,  graduated  from  the  University  of  Notre  Dame, 
South  Bend,  Indiana,  in  May  of  1982.  He  completed  the  Infantry  Officer  Basic  Course  at  Ft  Benning,  GA, 
and  was  assigned  as  Second  Lieutenant  to  a  reinforced  heavy  brigade  in  southern  Germany.  Lumb 
spent  the  next  10  years  in  various  Infantry  assignments,  alternating  between  Mechanized  and  Light 
Infantry  Divisions.  In  1991,  he  transition  from  the  Infantry  Branch  to  the  newly  formed  Army  Acquisition 
Corps,  spending  the  remainder  of  his  Army  career  in  various  acquisition  assignments,  including  anti-tank 
missiles,  ground  robotics  and  rotary-winged  aircraft.  While  in  the  Acquisition  Corps,  he  earned  a  Master’s 
Degree  in  Acquisition  and  Contract  Management  from  the  Naval  Postgraduate  School,  Monterey,  CA,  in 
1991.  Lumb  culminated  his  Army  Acquisition  career  as  the  inaugural  Program  Manager  of  the  Army’s 
UH-60M,  Blackhawk  helicopter  improvement  program.  After  retiring  from  the  Army  in  2002,  Lumb  joined 
the  Defense  Acquisition  University  where  he  served  as  a  member  of  the  faculty  in  the  Acquisition 
Program  Management  Department  and,  most  recently,  in  his  current  position  as  the  Director  of  Program 
Development. 

Dr.  Paul  A.  Alfieri 
Director  of  Research 
Defense  Acquisition  University 
Fort  Belvoir,  VA  22060 
Phone:  (703)  805-5282 

Mr.  Mark  D.  Lumb 

Director  of  Program  Development 

Defense  Acquisition  University — South  Region 

Huntsville,  AL  35806 

Phone:  (256)  722-1072 


Abstract 

The  Department  of  Defense  (DoD)  Acquisition  Process  has  been  the  subject  of  many 
reform  initiatives  and  improvement  attempts  over  the  last  several  decades.  The  acquisition 
process  is  deliberately  structured  for  oversight  (checks  and  balances)  and  decentralized  control. 
Separate,  independent  offices/staffs  within  the  DoD  make  requirements  determination,  resource 
allocations,  and  programmatic  decisions  (milestone  decisions).  Coupled  with  inter-service 
competition  for  missions  and  dollars,  this  process  involves  stakeholders  with  competing 
interests  and  is  certainly  not  designed  for  efficiency.  Rapid  growth  in  technology,  foreign 
competition  for  military  systems,  and  changing  threats  further  exacerbate  the  problem.  The 
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results  can  usually  be  categorized  as:  too  many  requirements,  too  many  programs,  unstable 
budgets,  insufficient  funds,  ever-changing  requirements. 

Most  stakeholders  in  the  process  have  much  to  gain  and  much  to  lose.  Whether  we  talk 
in  terms  of  political  dominance,  global  power,  military  deterrence  or  maximization  of  budget 
share,  the  stakes  of  the  game  are  very  high.  This  fact  is  certainly  not  lost  on  Congress.  In 
Public  Law  ' 109-163 ,  FY06  National  Defense  Authorization  Act  (Section  814),  Congress 
directed  the  Defense  Acquisition  University  (DAU),  acting  under  the  direction  and  authority  of 
USD(AT&L),  to  conduct  a  major  review  of  all  DoD  acquisition  organizations.  From  this 
legislation,  it  appears  that  Congress  desires  more  control  over  the  acquisition  process  and, 
more  importantly,  improved  acquisition  outcomes. 

While  conducting  the  Defense  Acquisition  Structures  &  Capabilities  Review,  the  DAU 
was  asked  to  specifically  examine  the  structure  of  the  DoD’s  acquisition  organizations  and  to 
address  the  capabilities  and  capacities  that  acquisition  organizations  require  to  successfully 
discharge  their  acquisition  missions.  This  study  will  provide  an  additional  examination  of 
capability  shortfalls  and  gaps,  along  with  appropriate  recommendations  for  reform  and 
improvement. 

Employing  a  survey-instrument  methodology,  utilizing  both  written  responses  and  face- 
to-face  interviews,  and  gathering  data  from  the  Military  Departments,  the  Defense  Agencies, 
Field  Operating  Activities  and  numerous  key  leaders  throughout  the  Department  of  Defense, 
this  paper  will  attempt  to  explore  the  relationship  between  organizational  re-structuring, 
acquisition  capabilities  and  capacities,  and  positive  acquisition  outcomes. 
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An  Integrated  Portfolio  Management  Approach  to  Weapon  System 
Investments  Could  Improve  DoD’s  Acquisition  Outcomes 


Presenter:  Katherine  Schinasi,  US  Government  Accountability  Office 


What  GAO  Found 

To  achieve  a  balanced  mix  of  executable  development  programs  and  ensure  a  good 
return  on  their  investments,  the  successful  commercial  companies  GAO  reviewed  take  an 
integrated,  portfolio  management  approach  to  product  development.  Through  this  approach, 
companies  assess  product  investments  collectively  from  an  enterprise  level,  rather  than  as 
independent  and  unrelated  initiatives.  They  weigh  the  relative  costs,  benefits,  and  risks  of 
proposed  products  using  established  criteria  and  methods,  and  select  those  products  that  can 
exploit  promising  market  opportunities  within  resource  constraints  and  move  the  company 
toward  meeting  its  strategic  goals  and  objectives.  Investment  decisions  are  frequently  revisited, 
and  if  a  product  falls  short  of  expectations,  companies  make  tough  go/no-go  decisions.  The 
companies  GAO  reviewed  have  found  that  effective  portfolio  management  requires  a 
governance  structure  with  committed  leadership,  clearly  aligned  roles  and  responsibilities, 
portfolio  managers  who  are  empowered  to  make  investment  decisions,  and  accountability  at  all 
levels  of  the  organization. 

In  contrast,  DOD  approves  proposed  programs  with  much  less  consideration  of  its 
overall  portfolio  and  commits  to  them  earlier  and  with  less  knowledge  of  cost  and  feasibility. 
Although  the  military  services  fight  together  on  the  battlefield  as  a  joint  force,  they  identify  needs 
and  allocate  resources  separately,  using  fragmented  decision-making  processes  that  do  not 
allow  for  an  integrated,  portfolio  management  approach  like  that  used  by  successful  commercial 
companies.  Consequently,  DOD  has  less  assurance  that  its  investment  decisions  address  the 
right  mix  of  warfighting  needs,  and,  as  seen  in  the  figure  below,  it  starts  more  programs  than 
current  and  likely  future  resources  can  support,  a  practice  that  has  created  a  fiscal  bow  wave.  If 
this  trend  goes  unchecked,  Congress  will  be  faced  with  a  difficult  choice:  pull  dollars  from  other 
high-priority  federal  programs  to  fund  DOD’s  acquisitions  or  accept  gaps  in  warfighting 
capabilities. 
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Figure:  Costs  Remaining  Versus  Annual  Appropriations  for  Major  Defense  Acquisitions 
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Source.  DOD  (data):  GAOisnafysfe  arid  pesentaucn). 


Figure  1.  Costs  Remaining  vs.  Annual  Appropriations  for  Major  Defense  Acquisitions 


Why  GAO  did  this  Study 

Over  the  next  several  years,  the  Department  of  Defense  (DOD)  plans  to  invest  $1.4 
trillion  in  major  weapons  programs.  While  DOD  produces  superior  weapons,  GAO  has  found 
that  the  department  has  failed  to  deliver  weapon  systems  on  time,  within  budget,  and  with 
desired  capabilities.  While  recent  changes  to  DOD’s  acquisition  policy  held  the  potential  to 
improve  outcomes,  programs  continue  to  experience  significant  cost  and  schedule  overruns. 

GAO  was  asked  to  examine  how  DOD’s  processes  for  determining  needs  and  allocating 
resources  can  better  support  weapon  system  program  stability.  Specifically,  GAO  compared 
DOD’s  processes  for  investing  in  weapon  systems  to  the  best  practices  that  successful 
commercial  companies  use  to  achieve  a  balanced  mix  of  new  products,  and  identified  areas 
where  DOD  can  do  better.  In  conducting  its  work,  GAO  identified  the  best  practices  of: 
Caterpillar,  Eli  Lilly,  IBM,  Motorola,  and  Procter  and  Gamble. 

What  GAO  Recommends 

GAO  is  making  several  recommendations  for  DOD  to  implement  an  integrated  portfolio 
management  approach  to  weapon  system  investments.  DOD  stated  that  it  is  undertaking 
several  pilot  efforts  to  improve  the  department’s  approach  and  that  implementation  of  any  new 
business  rules  will  be  contingent  upon  the  outcomes  of  these  efforts. 

www.qao.qov/cqi-bin-qetrpt?GAO-07-388 

To  view  the  full  product,  including  the  scope  and  methodology,  click  on  the  link  above. 

For  more  information,  contact  Michael  J.  Sullivan  at  (202)  512-4841  or  sullivanm@gao.gov. 
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Abstract 

The  specific  purpose  of  the  research  was  to  review  current  policies,  practices,  and 
procedures  for  funding  and  managing  Contract  Termination  Liability  within  the  Department  of 
Defense  (DoD).  The  research  proposes  alternative  approaches  for  improving  the  DoD’s  ability 
to  manage  Contract  Termination  Liability  and  discusses  the  resulting  effect  of  each  alternative 
on  defense  acquisition  practices.  First,  we  provide  a  brief  review  of  regulatory  and  policy 
guidance  on  Contract  Termination  Liability  as  reflected  in  the  Federal  Acquisition  Regulation 
(FAR)  and  the  Financial  Management  Regulations  ( FMR ).  We  then  discuss  the  current 
practices  and  procedures  for  funding  and  managing  Contract  Termination  Liability.  Next,  we 
present  program  management  challenges  and  observations  and  findings  based  on  our  research 
of  current  Contract  Termination  Liability  policies  and  real-world  practices.  A  discussion  of 
alternative  approaches  to  funding  Contract  Termination  Liability  is  then  presented,  including  the 
use  of  Special  Termination  Cost  Clauses  (STCC).  Finally,  this  research  concludes  with  a 
summary  and  recommendations  on  how  the  DoD  could  improve  the  policies  and  practices  for 
managing  Contract  Termination  Liability. 

A  copy  of  the  complete  report  is  available  at  the  following  website:  www.acquisitionresearch.org: 
Report  Number  NPS-CM-06-042 
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Executive  Summary 

This  research  paper  explores  the  Department  of  Defense  (DoD)  policies  and  practices 
for  managing  Contract  Termination  Liability.  The  specific  purpose  of  the  research  was  to  review 
current  policies,  practices,  and  procedures  for  funding  and  managing  Contract  Termination 
Liability  within  the  DoD.  Alternative  approaches  for  improving  the  DoD’s  ability  to  manage 
Contract  Termination  Liability  are  proposed  and  the  resulting  effect  of  each  alternative  on 
defense  acquisition  practices  is  discussed.  Recommendations  on  how  the  DoD  could  improve 
the  policies  and  practices  for  managing  Contract  Termination  Liability  are  provided. 

This  research  found  that  the  regulations  and  policies  pertaining  to  the  management  and 
funding  of  Contract  Termination  Liability  are  inconsistent  and  subject  to  interpretation.  Program 
managers,  finance  and  budget  managers,  and  contracting  officers  have  differing  interpretations 
of  the  requirement  for  funding  Contract  Termination  Liability.  Furthermore,  the  practices  and 
procedures  used  in  defense  acquisition  program  offices  reflect  this  inconsistency. 

A  review  of  current  practices  and  procedures  for  funding  and  managing  Contract 
Termination  Liability  and  historical  data  of  past  contract  terminations  found  that  the  probability  of 
a  contract  termination  for  convenience  is  very  small,  and  program  managers’  approaches  to 
managing  Contract  Termination  Liability  reflects  this  probability.  The  normal  procedure  for 
handling  the  potential  liability  associated  with  a  contract  terminated  for  convenience  is  to 
“budget”  for  the  liability.  Then,  in  coordination  with  the  contractor,  the  required  amount  of 
funding  is  tracked  on  a  regular  basis.  In  this  case,  budgeting  for  Termination  Liability  does  not 
mean  obligating  funds  specifically  for  that  purpose. 

Interviews  with  various  acquisition  program  offices  indicate  that  program  managers  are 
generally  satisfied  with  the  current  method  for  managing  Contract  Termination  Liability  because 
the  procedure  they  currently  use  allows  them  to  keep  all  of  the  funding  appropriated  for  their 
program.  Furthermore,  program  managers  are  not  in  favor  of  a  “tax”  that  would  negate  the 
requirement  to  budget  for  Contract  Termination  Liability.  A  tax  would  deprive  them  of  funds  that 
they  currently  have  at  their  disposal.  Additionally,  if  all  programs  were  taxed,  there  is  a  general 
concern  that  the  pooled  funds  would  likely  be  lost — either  the  Military  Departments  (or  DoD) 
would  use  them  to  solve  other  problems  if  they  were  not  required  to  cover  a  liability,  or 
Congress  would  look  upon  the  funds  that  had  been  set  aside  as  a  “slush  fund,”  making  them 
tempting  for  other  uses. 

Interviews  also  indicated  that  support  for  increased  use  of  STCCs  is  not  evident,  either 
at  the  program  level  or  the  OMB  or  Congressional  level.  Congress  has  expressed  its  concern 
through  report  language.  OMB  correspondence  has  indicated  that  support  for  more  than  one 
STCC  per  service  is  unlikely.  However,  it  should  be  noted  that  those  programs  that  have 
significant  funding  problems  and/or  are  concerned  about  the  possibilities  of  termination  do 
support  additional  use  of  STCCs.  In  fact,  these  programs  would  prefer  to  have  a  STCC  that 
covers  more  cost  elements  than  the  standard  STCC. 

Finally,  this  research  concluded  with  the  following  recommendations  for  the  DoD’s 
management  of  Contract  Termination  Liability:  1 .  Remove  the  ambiguity  and  improve  the 
consistency  in  the  regulations  pertaining  to  the  management  of  Contract  Termination  Liability;  2. 
Refrain  from  imposing  a  tax  system  to  provide  funding  for  potential  Contract  Termination 
Liability,  and  3.  Continue  to  use  STCCs  for  the  larger  programs  with  funding  or  longevity 
concerns. 
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Regulatory  and  Policy  Guidance 

This  section  of  the  research  report  focuses  on  the  regulatory  and  policy  guidance  on 
Termination  Liability  and  the  Special  Termination  Cost  Clause  (STCC).  The  regulatory  and 
policy  guidance  covering  Termination  Liability  (and,  specifically,  Special  Termination  Cost 
Clauses  (STCC))  is  found  in  the  DoD  Financial  Management  Regulation  ( FMR )  and  the 
Defense  Federal  Acquisition  Regulation  Supplement  ( DFARS ).  In  addition,  the  Air  Force 
Financial  Management  Regulation  is  also  discussed  as  an  example  of  Agency-specific  guidance 
on  contingent  liability. 


Termination  Liability 

The  DoD  Financial  Management  Regulation  (FMR)  defines  Termination  Liability  as: 

The  amount  of  prepayments  that  cover  payments  required  by  the  contract,  and  any 
damages  and  costs  that  may  accrue  from  the  cancellation  of  such  contract.  Funds 
prepaid  for  Termination  Liability  will  convert  to  cover  actual  expenditures  in  the  event 
that  the  contract  not  be  terminated  prior  to  performance  completion.  Termination  Liability 
may  not  apply  to  articles/services  provided  under  other  authorities  of  the  Foreign 
Assistance  Act  or  AECA.  (DoD,  2006c,  Vol.  15) 

The  Financial  Management  Regulation  (FMR)  categorizes  Contingent  Liabilities  (CLs)  as 
probable,  possible,  or  remote  (DoD,  2006c).  The  terms  probable,  reasonably  possible,  and 
remote  identify  three  areas  within  that  range  as  follows: 

1 .  Probable:  The  future  event  or  events  are  likely  to  occur. 

2.  Reasonably  possible:  The  chance  of  the  future  event  or  events  occurring  is  more  than 
remote  but  less  than  likely. 

3.  Remote:  The  chance  of  the  future  event  or  events  occurring  is  slight. 

Probable  CLs  must  be  covered  by  a  commitment  of  funds.  Probable  CLs  are  most  likely 
to  become  actual  liabilities.  Commitments  are  not  required  for  possible  CLs  and  should  not  be 
established  for  remote  CLs  (DoD,  2006c,  Vol.  4,  Ch.  13,  pp.  241-242). 

The  budgeting  for  Contingent  Liabilities  is  discussed  in  the  following  excerpts  taken  from 
the  DoD  Financial  Management  Regulation : 


Special  Provisions  for  Determining  the  Amounts  of  Commitments 

Contingent  Liabilities  Remaining  Under  Outstanding  Contracts.  There  are 
contingent  liabilities  for  price  or  quantity  increases  or  other  variables  that  cannot  be 
recorded  as  valid  obligations  in  the  cases  of  (1)  outstanding  fixed-price  contracts 
containing  escalation,  price  redetermination,  or  incentive  clauses,  or  (2)  contracts 
authorizing  variations  in  quantities  to  be  delivered,  or  (3)  contracts  where  allowable 
interest  may  become  payable  by  the  US  Government  on  contractor  claims  supported  by 
written  appeals  pursuant  to  the  “Disputes”  clause  contained  in  the  contract  (see 
subparagraph  080202. D,  below).  Amounts  to  cover  these  contingent  liabilities  should  be 
carried  as  outstanding  commitments  pending  determination  of  actual  obligations.  The 
amounts  of  such  contingent  liabilities,  however,  need  not  be  recorded  at  the  maximum  or 
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ceiling  prices  under  the  contracts.  Rather,  amounts  should  be  committed  that  are 
estimated  conservatively  to  be  sufficient  to  cover  the  additional  obligations  that  probably 
will  materialize,  based  upon  judgment  and  experience.  In  determining  the  amount  to  be 
committed,  allowances  may  be  made  for  the  possibility  of  downward  price  revisions  and 
quantity  underruns.  Each  contingent  liability  shall  be  supported  by  sufficient  detail  to 
facilitate  audit.  (DoD,  2006c,  Vol.  3,  Ch.  8,  para.  080202) 


Budgeting  for  Termination  Liability  on  Incrementally  Funded  RDT&E 
Contracts 

The  legal  requirements  of  the  Anti-deficiency  Act  and  the  long-standing  policy  of  not 
committing  a  successor  Congress  to  a  course  of  action  both  make  it  necessary  that  the 
unliquidated  obligation  for  an  incrementally  funded,  multiple-year  contract  be  sufficient  at  all 
times  to  cover  the  cost  of  terminating  that  contract  for  the  convenience  of  the  Government. 

Budgeting  to  cover  Termination  Liability  will  not  increase  the  total  amount  budgeted  for 
the  program.  It  will  require  that  the  distribution  of  funds  by  fiscal  year  be  shifted  more  towards 
the  earlier  years  of  the  contract  than  if  funds  had  been  budgeted  only  to  cover  the  actual  bill  to 
be  paid  each  year.  The  distribution  of  funds  by  fiscal  year  shall  be  such  that,  if  a  contract  is 
terminated  at  any  point  during  the  fiscal  year,  all  termination  costs  can  be  financed  from  the 
unliquidated  obligation  on  the  contract  without  recourse  to  reprogramming  of  funds, 
supplemental  appropriations,  or  awaiting  the  appropriation  of  funds  for  the  succeeding  fiscal 
year’s  funding  increment.  All  programs  shall  adhere  to  this  policy  with  the  following  two 
exceptions,  both  of  which  are  to  be  used  rarely. 

a.  Special  Termination  Cost  Clause  (STCC).  DoD  FAR  Parts  249.50170  and  252.249-7000 
permit  the  use  of  STCC  in  fixed-price  incentive  contracts  and  incrementally  funded  cost 
reimbursement  contracts.  If  contracts  containing  an  STCC  are  terminated  before 
completion,  the  special  termination  charges  are  covered  by  the  unobligated  balance  of 
the  applicable  appropriation,  subject  to  any  congressional  approval  required  for 
reprogramming.  The  extent  to  which  the  STCC  can  be  used  is  limited  to  the  ability  of  the 
Service  or  Agency  to  cover  expected  termination  costs  from  unobligated  balances.  A 
recordable  obligation  under  the  STCC  arises  when  the  contract  is  actually  terminated.  If 
a  proposed  STCC  would  require  an  above-threshold  reprogramming  action  when  a 
program  is  terminated,  the  approval  to  use  the  STCC  shall  be  obtained  from  the  USD 
(Comptroller)  before  the  contract  or  contract  modification  is  awarded.  All  STCCs, 
regardless  of  dollar  amount,  require  prior  notification  of  the  House  and  Senate 
Appropriations  Committees. 

b.  Statutory  Waivers.  The  Department  is  not  required  to  budget  for,  or  obligate  funds 
sufficient  to  cover,  Termination  Liability  in  connection  with  an  incrementally  funded 
RDT&E  contract  if  Congress  has  expressly  exempted  the  program  or  contract  from  that 
requirement.  When  this  situation  arises,  however,  the  budget  exhibits  for  the  program 
shall  clearly  indicate  the  value  of  the  unfunded  Termination  Liability  by  year  for  the 
current  year,  budget  year,  and  the  outyears  covered  by  the  FYDP.  (DoD,  2006c,  Vol.  2A, 
Ch.  1,  para.  010214) 

Termination  Liability  is  considered  a  contingent  liability  since  adequate  funds  must  be 
committed  to  cover  the  liabilities  resulting  from  the  termination  of  contracts,  including  any 
potential  or  Contingent  Liabilities  (Gill,  2003). 

The  DoD  FMR  explains  Contingent  Liabilities  as  follows: 
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Contingent  Liability — The  term  has  two  meanings.  As  a  budgetary  term,  it  represents 
variables  that  cannot  be  recorded  as  valid  obligations.  Such  variables  include  (1) 
outstanding  fixed-price  contracts  containing  escalation,  price  redetermination,  or 
incentive  clauses,  or  (2)  contracts  authorizing  variations  on  quantities  to  be  delivered,  or 
(3)  contracts  where  allowable  interest  may  become  payable  by  the  US  Government  on 
contractor  claims  supported  by  written  appeals  pursuant  to  the  "DISPUTES"  clause 
contained  in  the  contract.  As  a  proprietary  accounting  term,  it  represents  an  obligation, 
relating  to  a  past  transaction  or  other  event  or  condition  that  may  arise  in  consequence, 
as  a  future  event  now  deemed  possible  but  not  probable.  When  the  liability  is 
determined  to  be  possible,  but  not  probable,  the  potential  liability  is  disclosed  as  a 
footnote  to  the  financial  statements.  When  the  potential  liability  becomes  probable,  it  is 
recorded  in  the  accounts  as  a  current  liability  or  a  reduction  of  an  asset.  The  budget 
definition  is  the  preferred  usage.  (DoD,  2006c,  Vol.  15) 

Thus,  according  to  DoD  FMR,  Volume  2A,  Chapter  1,  "all  termination  costs  can  be 
financed  from  the  unliquidated  obligation  on  the  contract  without  recourse  to  reprogramming  of 
funds,  supplemental  appropriations,  or  awaiting  the  appropriation  of  funds  for  the  succeeding 
fiscal  year's  funding  increment"  (2006c).  The  two  exemptions  to  this  are  a  Special  termination 
Cost  Clause  (STCC)  and  a  Statutory  Waiver. 

In  addition,  Volume  3,  Chapter  8,  Section  080512  of  the  DoD  FMR  states  that  in  the 
case  of  termination  of  a  contract,  the  contract  shall  be  decreased  to  an  amount  that  is  sufficient 
to  meet  the  settlement  costs  under  the  termination. 

The  Air  Force  Material  Command  (AFMC)  Financial  Management  Reference  System 
(2005,  February)  provided  more  detailed  guidance  on  funding  termination  costs.  The  AFMC 
FMRS  states  the  following  concerning  funding  termination  costs: 

The  funded  activity  should  commit  the  estimated  funds  to  cover  the  expected  contingent 
liability  (CL).  This  estimated  CL  amount  is  in  excess  of  the  contract  awarded  amount 
recorded  as  an  obligation.  The  financial  manager  must  record  commitments  for  CLs 
against  the  applicable  FY  and  appropriation  cited  on  the  contract.  Normally,  funds  for 
CLs  are  maintained  locally.  Funds  are  committed  for  a  contingent  liability  at  the  time  of 
contract  award,  based  on  the  amount  provided  by  the  contracting  officer  [...]. 
Commitments  are  not  recorded  for  STCC  or  contingent  termination  liabilities. 

Obligations  are  recorded  when  the  action  to  terminate  is  taken.  (AFMC,  2005,  February) 

The  AFMC  FMRS  further  states  that  funds  are  committed  for  all  “probable”  CLs  (funding 
for  “possible”  or  “remote”  CLs  is  not  necessary)  as  defined  in  a  matrix.  “The  CL  Matrix  is  used 
to  identify,  categorize  according  to  probability,  and  track  CLs  throughout  the  life  of  a  contract  [...] 
must  be  reported  to  SAF/FM  semi-annually”  (AFMC,  2005,  February). 

As  indicated  above,  the  DoD  FMR  refers  to  two  exceptions  to  the  policy  of  budgeting  for 
Termination  Liability.  These  include  the  Special  Termination  Cost  Clause  (STCC)  and  the 
Statutory  Waiver.  These  will  be  discussed  below. 


Special  Termination  Cost  Clause  (STCC) 

Regulatory  and  policy  guidance  related  to  the  use  of  Special  Termination  Cost  Clauses 
is  found  in  the  DoD  FMR  (Section:  “Budgeting  for  Termination  Liability  on  Incrementally  Funded 
RDT&E  Contracts,”  p.  3)  and  the  DoD  FAR. 
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Although  the  Federal  Acquisition  Regulation  (FAR)  Part  49  provides  guidance  on 
contract  terminations,  the  Defense  FAR  Supplement  ( DFARS )  provides  the  guidance  and 
prescribes  the  clause  specifically  for  Special  Termination  Costs.  The  DFARS  guidance  at 
249.501-70  states  the  following: 

249.501-70  Special  Termination  Costs. 


(a)  The  clause  at  252.249-7000,  Special  Termination  Costs,  may  be  used  in  an 
incrementally  funded  contract  when  its  use  is  approved  by  the  agency  head. 

(b)  The  clause  is  authorized  when — 

(1 )  The  contract  term  is  two  years  or  more; 

(2)  The  contract  is  estimated  to  require — 

(i)  Total  RDT&E  financing  in  excess  of  $25  million;  or 

(ii)  Total  production  investment  in  excess  of  $100  million;  and 

(3)  Adequate  funds  are  available  to  cover  the  contingent  reserve  liability  for 
special  termination  costs. 

(c)  The  contractor  and  the  contracting  officer  must  agree  upon  an  amount  that 
represents  their  best  estimate  of  the  total  special  termination  costs  to  which  the 
contractor  would  be  entitled  in  the  event  of  termination  of  the  contract.  Insert  this 
amount  in  paragraph  I  of  the  clause. 

(d)  (1)  Consider  substituting  an  alternate  paragraph  I  for  paragraph  I  of  the  basic 
clause  when — 

(i)  The  contract  covers  an  unusually  long  performance  period;  or 

(ii)  The  contractor’s  cost  risk  associated  with  contingent  special 
termination  costs  is  expected  to  fluctuate  extensively  over  the  period 
of  the  contract. 

(2)  The  alternate  paragraph  I  should  provide  for  periodic  negotiation  and 

adjustment  of  the  amount  reserved  for  special  termination  costs.  Occasions 
for  periodic  adjustment  may  include — 

(i)  The  Government’s  incremental  assignment  of  funds  to  the  contract; 

(ii)  The  time  when  certain  performance  milestones  are  accomplished  by 
the  contractor;  or 

(iii)  Other  specific  time  periods  agreed  upon  by  the  contracting  officer  and 
the  contractor. 
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A  review  of  the  DFARS  clause  reveals  that  the  clause  may  be  used  on  incrementally 
funded  contracts  when:  the  contract  term  is  two  years  or  longer  and  is  estimated  to  require  in 
excess  of  $25  million  of  Research,  Development,  Test,  and  Evaluation  (RDT&E)  funds  or  a  total 
of  over  $100  million  of  production  investment. 

Incrementally  funded  contracts  are  those  contracts  in  which  funds  are  incrementally 
obligated  throughout  the  period  of  performance.  Typically,  cost  reimbursement  RDT&E 
contracts  are  incrementally  funded  and  require  the  use  of  the  Limitation  of  Funds  Clause  at  FAR 
52.232-22.  This  clause  requires  the  contractor  to  notify  the  Contracting  Officer  in  writing 
whenever  it  has  reason  to  believe  the  cost  it  expects  to  incur  in  the  next  60  days,  when  added  to 
all  costs  previously  incurred,  will  exceed  75%  of  the  total  amount  allotted  on  the  contract  (DoD, 
2006b,  52.232-22). 

Another  requirement  of  the  Special  Termination  Cost  Clause  (STCC)  is  that  there  will  be 
adequate  funds  available  to  cover  the  contingent  reserve  liability  for  special  termination  costs. 

In  addition,  the  clause  states  that  the  contractor  and  the  contracting  officer  must  agree 
upon  an  amount  that  represents  their  best  estimate  of  the  total  special  termination  costs  to 
which  the  contractor  would  be  entitled  in  the  event  of  termination  of  the  contract.  These  special 
termination  costs  are  identified  within  the  DFARS  in  the  actual  Special  Termination  Costs  clause 
as  follows: 

252.249-7000  Special  Termination  Costs. 


As  prescribed  in  249.501-70,  use  the  following  clause: 


SPECIAL  TERMINATION  COSTS  (DEC  1991) 


(a)  Definition.  “Special  termination  costs,”  as  used  in  this  clause,  means  only  costs  in  the 
following  categories  as  defined  in  Part  31  of  the  Federal  Acquisition  Regulation 

(■ FAR )~ 

(1)  Severance  pay,  as  provided  in  FAR  31.205-6(g); 

(2)  Reasonable  costs  continuing  after  termination,  as  provided  in  FAR  31.205- 
42(b); 

(3)  Settlement  of  expenses,  as  provided  in  FAR  31.205-42(g); 

(4)  Costs  of  return  of  field  service  personnel  from  sites,  as  provided  in  FAR 
31 .205-35  and  FAR  31 .205-461;  and 

(5)  Costs  in  paragraphs  (a)(1 ),  (2),  (3),  and  (4)  of  this  clause  to  which 
subcontractors  may  be  entitled  in  the  event  of  termination. 

(b)  Notwithstanding  the  Limitation  of  Cost/Limitation  of  Funds  clause  of  this  contract,  the 
Contractor  shall  not  include  in  its  estimate  of  costs  incurred  or  to  be  incurred,  any 
amount  for  special  termination  costs  to  which  the  Contractor  may  be  entitled  in  the 
event  this  contract  is  terminated  for  the  convenience  of  the  Government. 
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(c)  The  Contractor  agrees  to  perform  this  contract  in  such  a  manner  that  the  Contractor’s 

claim  for  special  termination  costs  will  not  exceed  $ _ .  The  Government  shall 

have  no  obligation  to  pay  the  Contractor  any  amount  for  the  special  termination  costs 
in  excess  of  this  amount. 

(d)  In  the  event  of  termination  for  the  convenience  of  the  Government,  this  clause  shall 
not  be  construed  as  affecting  the  allowability  of  special  termination  costs  in  any 
manner  other  than  limiting  the  maximum  amount  of  the  costs  payable  by  the 
Government. 

(e)  This  clause  shall  remain  in  full  force  and  effect  until  this  contract  is  fully  funded.  (End 
of  clause)(DoD,  2006a,  252.249-7000) 

Thus,  the  Special  Termination  Cost  Clause  limits  the  amount  of  special  termination  (as 
agreed  between  the  government  and  the  contractor)  costs  that  the  Government  is  liable  for  in  a 
Termination  for  Convenience.  It  should  be  noted  that  the  STC  clause  does  not  apply  to  the 
regular  termination  costs  as  outlined  in  FAR  31.205-42. 

Agency  Approval  for  STCC 

As  stated  in  the  DFARS  clause,  the  use  of  the  STC  clause  is  subject  to  approval  of  the 
agency  head.  A  review  of  the  various  agency  FAR  supplements  provides  some  perspective  on 
how  this  approval  is  obtained. 

The  Air  Force  FAR  supplement  at  AFFARS  5349.501-70  provides  additional  and  specific 
policy  related  to  the  use  of  the  Special  Termination  Cost  Clause.  AFFARS  5349.501-70 
specifically  states  the  following: 

5349.501-70  Special  termination  costs. 

(a)  Contracting  officers  shall  refer  to  Volume  2A,  Chapter  1,  Section  010213,  paragraph 
C.2  of  DoD  7000. 14-R,  DoD  Financial  Management  Regulation ,  for  Congressional 
notification  and  additional  approval  requirements  for  Special  Termination  Cost 
Clauses  (STCCs).  Because  STCCs  require  special  notification  to  Congress  and 
entail  a  long  approval  process  over  which  the  Air  Force  has  little  control,  the 
contracting  officer  should  allow  SAF/AQCK  sufficient  time  to  process  requests  to  use 
DFARS  252.249-7000,  Special  Termination  Costs  (i.e.,  not  less  than  90  days  prior  to 
contract  award).  The  request  shall  include  the  following: 

(i)  A  detailed  breakdown  of  applicable  cost  categories  in  the  clause  at  DFARS 
252.249-7000  (a)(1)  through  (5),  which  includes  the  reasons  for  the 
anticipated  incurrence  of  the  costs  in  each  category; 

(ii)  Information  on  the  financial  and  program  need  for  the  clause,  including  an 
assessment  of  the  contractor’s  financial  position  and  the  impact  of  a  failure  to 
receive  authority  to  use  the  clause;  and 

(iii)  Clear  evidence  that  only  costs  that  arise  directly  from  a  termination  would  be 
compensated  under  the  clause.  Costs  that  would  be  incurred  by  the 
Government,  regardless  of  whether  a  termination  occurs,  shall  not  be  covered 
by  an  STCC. 
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(b)  The  contracting  officer  shall  obtain  SAF/FM  approval  prior  to  authorizing  any 

increase  in  the  Government’s  maximum  liability  under  the  clause.  (Air  Force,  2006, 
5349.501-70) 

The  AFFARS  is  the  only  agency-level  FAR  guidance  that  gives  more  specific  instruction 
on  the  coordination  and  review  process,  as  well  as  on  the  Congressional  notification 
requirement  for  the  use  of  STCCs.  This  guidance  also  identifies  the  requirement  for  referencing 
the  DoD  Financial  Management  Regulations  ( FMR )  for  specific  notification  and  approval 
requirements. 


Statutory  Waiver 

The  second  exception  to  the  Termination  Liability  funding  policy  is  the  Statutory  Waiver. 
This  exception  is  explained  in  the  FMR  as  follows: 

Statutory  Waivers.  If  a  program  is  exempted  by  Public  Law  from  the  requirement  to 
budget  for  Termination  Liability,  the  fiscal  year  increments  may  be  budgeted  on  a  pay- 
as-you-go  basis,  providing  only  sufficient  funds  to  cover  the  disbursements  expected  to 
be  made  in  that  fiscal  year.  When  this  situation  arises,  however,  the  budget  exhibits  for 
the  program  shall  clearly  indicate  the  value  of  the  unfunded  Termination  Liability  by  year 
for  the  current  year,  budget  year,  and  the  outyears  covered  by  the  FYDP.  (DoD,  2006c) 

As  can  be  seen  from  the  above  discussion,  the  regulatory  and  policy  guidance  pertaining 
to  the  funding  of  Termination  Liability  and  the  use  of  STCCs  is  found  in  two  different  functionally 
oriented  regulations — the  Financial  Management  Regulation  (FMR)  and  the  Federal  Acquisition 
Regulation  (FAR).  This  regulatory  guidance  on  budgeting  for  Contract  Termination  Liability  from 
two  different  functional  areas  of  DoD  acquisition  increases  the  potential  for  different 
interpretations  or  even  misinterpretation  of  the  DoD  policy.  These  differences  in  policy 
interpretation  are  reflected  in  the  practices  and  procedures  used  by  the  various  DoD  services. 

Observations  and  Findings 

The  researchers  conducted  interviews  with  various  DoD  program  management  offices 
and  analyzed  samples  of  DoD  contracts  related  to  the  management  of  Termination  Liability. 
Based  on  these  reviews,  interviews,  and  analyses,  the  research  team  identified  the  following 
observations  and  findings: 

1.  Inconsistent  Approach 

There  is  an  inconsistent  approach  among  the  various  military  and  DoD  agencies  to 
managing  Termination  Liability  funds  on  contracts.  Although  all  program  offices  that  were 
interviewed  in  this  research  manage  Termination  Liability  based  on  the  funds  obligated  on 
contract,  the  procedures  used  for  ensuring  the  obligated  funds  are  adequate  and  sufficient  to 
cover  Termination  Liability  expense  at  any  point  during  the  contract  period  of  performance 
varied.  Some  program  offices  maintained  close  coordination  with  their  contractors  to  monitor 
and  ensure  sufficient  obligated  funds  to  cover  estimated  Termination  Liability  expenses 
throughout  the  contract  period,  while  other  program  offices  depended  solely  on  the  contractor  to 
monitor  the  obligated  funds  to  ensure  sufficient  coverage  for  Termination  Liability.  Some 
program  offices  conducted  periodical  “budget  drills”  to  determine  if  the  amount  of  obligated 
funds  at  any  given  time  would  be  sufficient  to  cover  the  estimated  Termination  Liability  at  that 
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point  in  time.  Some  program  offices  used  the  Contractor  Funds  Status  Report  (CFSR)  as  an  aid 
in  monitoring  the  estimated  Termination  Liability  expenses. 

2.  Diffused  Guidance 

The  regulatory  and  policy  guidance  pertaining  to  Termination  Liability  are  diffused 
between  the  Federal  Management  Regulation  ( FMR )  and  the  Federal  Acquisition  Regulation 
(FAR).  The  FMR  is  the  main  source  of  financial  management  policy  and  guidance  used  by  DoD 
financial  and  budget  managers,  while  the  FAR  is  the  main  source  for  contract  management 
policy  and  guidance  used  almost  exclusively  by  DoD  contracting  officers.  These  two 
functionally  based  regulations  lead  to  differing  interpretations  of  policy,  guidance,  and 
procedures  related  to  the  management  of  Termination  Liability  by  the  financial-management 
and  contract-management  functional  areas. 

3.  Insufficient  Databases 

There  is  no  DoD-wide,  Service-wide,  Command-wide,  or  Center-wide  database;  yet,  one 
is  needed  to  conduct  a  proper  analysis  to  determine  the  total  number  of  contracts  that  require 
funding  for  Termination  Liability,  the  total  amount  of  Termination  Liability  funding  on  these 
contracts,  the  total  number  of  contracts  containing  a  Special  Termination  Cost  (STC)  clause, 
and  the  total  amount  of  estimated  Termination  Liability  expenses  being  managed  at  the  Service 
levels  because  of  these  STC  clauses.  These  databases  would  provide  the  data  that  would  be 
considered  a  critical  part  of  the  business  case  needed  to  calculate  the  extent  of  the  funding 
being  budgeted  for  Termination  Liability  expenses. 

4.  Declining  Acceptability  of  Special  Termination  Cost  Clause 

Because  of  the  current  acquisition  climate  of  defense  acquisition  program  cost  overruns 
and  schedule  delays,  the  increased  use  of  the  current  Special  Termination  Cost  Clause  (STCC) 
would  not  be  well  received  by  the  Congress  or  the  Office  of  Management  and  Budget  (OMB). 
Furthermore,  program  managers  are  not  necessarily  receptive  to  requesting  approval  of  a 
STCC  from  their  higher  headquarters. 

Alternative  Approaches  to  Funding  Termination  Liability 

Our  research  identified  the  following  alternative  approaches  to  managing  and  funding 
contract  termination  Liability. 

1.  Impose  a  “Tax”  on  All  Programs  Subject  to  Termination  Liability  for  the 

Purpose  of  Establishing  an  Insurance  Fund  to  Cover  Termination  Liability. 

The  advantages  of  this  alternative  include  the  benefit  for  program  managers  of  not 
having  to  commit  funds  to  cover  TL,  thus  allowing  better  use  of  funds  for  program  execution. 
Additionally,  since  the  required  Termination  Liability  funds  would  be  identified  prior  to  any 
termination,  any  concerns  for  possible  Anti-deficiency  Act  violations  should  subside.  Finally,  for 
the  Military  department,  significantly  fewer  dollars  would  be  tied  up  unproductively  for  TL  and 
would  be  available  for  program  execution. 

The  disadvantages  of  this  option  include  the  fact  that  those  programs  not  at  risk  for 
termination  would  have  to  pay  this  TL  tax,  thus  decreasing  their  amount  of  budget  for  executing 
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the  program.  For  not-at-risk  programs,  this  tax  would  make  program  management  more 
difficult.  The  dollars  associated  with  this  tax  would  not  be  available  until  late  in  the  fiscal  year  if 
they  were  not  used  to  cover  a  termination;  if  they  were  used  to  cover  a  termination,  the  program 
would  lose  the  money  permanently — presenting  a  lose-lose  proposition  for  the  program 
manager.  Finally,  another  disadvantage  would  be  that  at-risk  programs  would  not  have  the 
funds  required  to  pay  for  the  tax  available  for  program  execution,  thus,  putting  these  programs 
at  an  increased  disadvantage. 

Some  of  the  potential  questions  related  to  this  alternative  include  the  following: 

•  Who  determines  the  “tax”?  Those  programs  at  greatest  risk  should  logically  be  taxed 
more  than  those  programs  not  at  risk. 

•  Who  determines  the  risk  of  a  possible  program  termination? 

•  Would  the  insurance  fund  provide  an  attractive  target  for  Congressional  rescissions  as 
well  as  Department  reprogrammings? 

•  When  and  how  would  the  unused  portion  of  the  funds  be  returned  to  the  programs? 

2.  Allow  Coverage  of  Termination  Liability  to  be  Assumed  at  the  Major  Command 
or  PEO  Level. 

One  advantage  of  this  alternative  is  that  program  managers  could  use  all  of  the  funds 
appropriated  for  their  programs  for  program  execution.  Additionally,  the  use  of  STCCs  with  the 
associated  Congressional  notification  would  not  be  required.  Another  advantage  of  this 
approach  is  that  the  uncertainty  of  fund  availability  (as  opposed  to  the  tax  approach)  would  be 
eliminated.  Finally,  there  would  not  be  a  pot  of  funds  to  be  targeted  by  Congress  or  the 
Department. 

The  disadvantages  of  this  option  include  the  fact  that  this  approach  is  similar  to  the 
STCC  approach — which  has  not  enjoyed  strong  support  from  the  OMB  or  the  Congress. 
Additionally,  concerns  regarding  possible  Anti-deficiency  Act  violations  would  likely  increase. 
Finally,  another  disadvantage  would  include  the  fact  that  paying  for  a  program’s  termination 
costs  would  likely  adversely  impact  other  programs. 

Some  of  the  potential  issues  related  to  this  alternative  include  the  following: 

•  This  approach  would  appear  to  the  OMB  and  Congress  as  an  attempt  to  forego 
budgeting  for  Termination  Liability. 

•  A  program  termination  late  in  the  fiscal  year  could  be  difficult  to  fund.  Above-threshold 
reprogramming  requests  are  rarely  certain  or  timely. 

3.  Increase  the  Use  of  Special  Termination  Cost  Clauses  (STCC) 

The  advantages  of  this  alternative  include  the  benefit  that  program  managers  would  be 
able  to  use  all  of  the  funds  appropriated  for  their  programs  for  program  execution.  The 
uncertainty  of  fund  availability  (as  opposed  to  in  the  tax  approach)  would  be  eliminated  for 
program  managers. 
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The  disadvantages  of  this  option  include  the  fact  that  Congress  and  the  OMB  have 
already  exhibited  a  lack  of  enthusiasm  for  the  increased  use  of  STCCs.  Additionally,  the 
paperwork  involved  with  STCCs  is  considered  onerous  by  the  programs  that  have  completed  it. 

Recommendations 

Based  on  the  research  findings,  the  following  recommendations  are  provided. 

1.  Remove  ambiguity  and  improve  consistency  in  the  regulations. 

The  current  regulations  pertaining  to  the  management  of  contract  termination  lend 
themselves  to  differing  and  inconsistent  interpretations  among  the  Services  and  functional  areas 
(program  management,  financial  management,  and  contract  management).  If  the  “liberal” 
interpretation  of  current  regulations  is  different  from  what  is  desired  or  is  the  intent  of  the 
agencies,  these  regulations  should  be  revised  to  remove  any  ambiguity  and  to  improve  the 
consistency  between  the  functional  areas. 

2.  Do  not  impose  a  tax  system  to  provide  funding  for  potential  Termination 
Liability. 

The  taxing  of  program  offices  for  the  purpose  of  generating  a  pool  of  funds  to  use  for 
Termination  Liability  results  in  a  lose-lose  proposition  for  program  offices  and  may  result  in  more 
disadvantages  than  advantages.  In  addition,  the  potential  issues  related  to  this  alternative 
would  require  additional  research  and  analysis. 

3.  Continue  to  use  STCCs  for  the  larger  programs  with  funding  or  longevity 
concerns. 

For  larger,  major  defense  acquisition  programs  that  have  a  lower  probability  of 
termination  due  to  visibility,  political  ties,  or  urgency  of  need,  the  DoD  should  continue  to 
support  the  use  of  STCCs  to  allow  for  greater  use  of  program  funds  for  program  execution. 

Summary  and  Conclusion 

The  purpose  of  this  research  was  to  explore  current  Department  of  Defense 
mechanisms  for  addressing  Contract  Termination  Liability,  review  current  practices  and 
procedures  for  funding  and  managing  Termination  Liability,  and  propose  alternative  approaches 
to  improve  the  DoD’s  ability  to  manage  Termination  Liability  and  its  effect  on  defense  acquisition 
programs.  This  research  reviewed  the  regulatory  and  policy  guidance  on  Contract  Termination 
Liability.  A  review  of  current  practices  and  procedures  for  funding  and  managing  Termination 
Liability  was  conducted  based  on  interviews  and  document  reviews  with  the  Air  Force,  Navy, 
and  other  various  DoD  agencies.  Program  management  challenges  and  preliminary 
observations  and  findings  were  then  presented.  A  discussion  of  alternative  approaches  to 
funding  Termination  Liability  was  discussed,  including  the  use  of  Special  Termination  Cost 
Clauses  (STCC).  Finally,  recommendations  were  presented. 

The  regulations  and  policies  pertaining  to  the  management  and  funding  of  Contract 
Termination  Liability  are  inconsistent  and  subject  to  interpretation.  Program  managers,  finance 
and  budget  managers,  and  contracting  officers  have  differing  interpretations  of  the  requirement 
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for  funding  Termination  Liability.  Furthermore,  the  practices  and  procedures  used  in  defense 
acquisition  program  offices  reflect  this  inconsistency. 

In  addition,  the  probability  that  a  government  contract  will  be  terminated  for  convenience 
is  very  small.  Program  managers  and  contractors  are  aware  of  the  statistics,  and  their  approach 
to  Termination  Liability  reflects  that  knowledge.  The  normal  procedure  for  handling  the  potential 
liability  associated  with  a  contract  terminated  for  convenience  is  to  “budget”  for  the  liability. 

Then,  in  coordination  with  the  contractor,  the  required  amount  of  funding  is  tracked  on  a  regular 
basis.  In  this  case,  budgeting  for  Termination  Liability  does  not  mean  obligating  funds 
specifically  for  that  purpose. 

Additionally,  program  managers  are  not  in  favor  of  a  “tax”  that  would  negate  the 
requirement  to  budget  for  TL.  For  the  most  part,  they  are  satisfied  with  the  status  quo  because 
the  procedure  they  currently  use  to  handle  TL  allows  them  to  keep  all  of  the  funding 
appropriated  for  their  program.  A  tax  would  deprive  them  of  funds  they  currently  have  at  their 
disposal.  In  fact,  a  program  that  has  funding  problems  could  be  put  in  jeopardy  by  having  to 
relinquish  funding  to  pay  for  a  tax.  Program  managers  feel  as  though  the  statistics  support  their 
current  approach. 

Furthermore,  if  all  programs  were  taxed,  there  is  a  general  concern  that  the  pooled  funds 
would  likely  be  lost  for  good — either  the  military  Departments  (or  DoD)  would  use  them  to  solve 
other  problems  if  they  were  not  required  to  cover  a  liability,  or  Congress  would  look  upon  the 
funds  that  had  been  set  aside  as  a  “slush  fund”  and  be  tempted  to  use  them  elsewhere. 

Also,  support  for  increased  use  of  STCCs  is  not  evident,  either  at  the  program  level  or 
the  OMB  or  Congressional  level.  Congress  has  expressed  its  concern  regarding  STCCs  through 
report  language.  OMB  correspondence  has  indicated  that  support  for  more  than  one  STCC  per 
service  is  unlikely.  However,  it  should  be  noted  that  those  programs  that  have  significant 
funding  problems  and/or  are  concerned  about  the  possibilities  of  termination  do  support 
additional  use  of  STCCs.  In  fact,  these  programs  would  prefer  to  have  a  STCC  that  covers  more 
cost  elements  than  the  standard  STCC. 

Finally,  this  research  recommended  that  the  Department  of  Defense:  remove  the 
ambiguity  and  improve  the  consistency  in  the  regulations  pertaining  to  the  management  of 
Termination  Liability,  not  impose  a  tax  system  to  provide  funding  for  potential  Termination 
Liability,  and  continue  to  use  STCCs  for  the  larger  programs  with  funding  or  longevity  concerns. 
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Abstract: 

Our  guiding  assumption  in  organization  of  our  research  and  this  report  is  that  to 
understand  the  defense  acquisition  process  and  reform  arguments,  it  is  necessary  to  know 
something  about  the  organizational  and  managerial  context  in  which  such  reform  must  take 
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place.  Consequently,  this  report  for  the  2007  NPS  Acquisition  Symposium  is  organized  into  four 
parts:  (i)  an  analysis  of  the  Enterprise  organization  and  management  initiative  now  underway  in 
Department  of  Defense  (DoD),  demonstrating  (ii)  how  it  encompasses  the  new  approach  to 
defense  capabilities  thinking,  planning  and  management  as  a  preamble  to  our  argument  for 
acquisition  system  and  process  reform.  Along  the  pathway  to  presentation  of  our  acquisition 
reform  proposals,  we  show  (iii)  the  role  of  better  business  practices  and  information  technology 
in  adding  value  to  DoD  acquisition  and  resource  management  in  terms  of  improved  organization 
strategy  based  on  lessons  from  economics  in  the  private  sector  in  evolving  from  bureaucracy  to 
hyperarchy  and  netcentric  organization.  It  this  section,  we  draw  lessons  from  the  manner  in 
which  businesses  operate  in  the  new  global  economy  and  how  the  development  of  new 
information  technology  should  enable  managerial  reform.  This  analysis  supports  the  types  of 
change  we  recommend  later  in  the  report  in  a  way  that  adds  value  to  DoD  acquisition  and 
resource  management.  We  advance  our  analysis  in  part  by  applying  lessons  from  economic, 
information  and  value-chain  theory  and  practice,  illustrating  the  utility  of  this  approach  using  the 
examples  of  Toyota  Motor  corporation  and  the  DoD  Global  Information  Grid  (GIG);  by 
employing  these  examples,  we  demonstrate  both  possibilities  and  obstacles  to  be  overcome  in 
reorganizing  the  DoD  and  its  acquisition  and  resource  management  processes  to  better  meet 
market  demand  and  to  respond  to  changes  in  the  threat  environment.  Part  of  this  argument 
includes  assessment  of  the  application  of  new  technology,  particularly  IT,  and  the  principle  of 
netcentricity  and  hyperarchy  in  DoD  reorganization  and  acquisition/resource  management 
reform.  We  assert  the  necessity  for  understanding  something  about  the  new  economics  of 
organizations  and  a  critique  of  bureaucratic  organization  as  critical  intellectual  components  of 
support  for  our  proposed  reforms.  Finally,  (iv)  we  advance  two  approaches  to  reform  in  terms  of 
magnitude  of  change  in  DoD  acquisition,  procurement  and  resource  management:  (a)  an 
argument  for  marginal  adjustment  based  on  our  view  of  the  need  for  implementation  of  longer- 
term  capital  and  performance-oriented  budgeting  in  combination  with  radical  DoD  business 
process  reengineering,  consistent  with  the  principles,  methods  and  goals  of  enterprise 
management,  and  (b)  a  much  more  radical  conversion  of  the  DoD  to  an  approach  that  we  term 
"marketization  and  privatization"  of  defense  acquisition  systems  and  resource-management 
processes.  We  note  that  these  options  are  not  mutually  exclusive,  as  both  are  needed. 


Preface 


Problem:  If  a  cop  in  Anytown,  USA,  pulls  over  a  suspect,  [ideally]  he  checks  the  person's 
ID  remotely  from  the  squad  car.  He's  linked  to  databases  filled  with  Who's  Who  in  the  world  of 
crime,  killing  and  mayhem.  In  Iraq,  there  is  nothing  like  that.  When  our  troops  and  the  Iraqi  army 
enter  a  town,  village  or  street,  what  they  know  about  the  local  bad  guys  is  pretty  much  in  their 
heads,  at  best.  Solution:  Give  our  troops  what  [some  of]  our  cops  have.  The  Pentagon  knows 
this.  For  reasons  you  can  imagine,  it  hasn't  happened...  This  is  a  story  of  can-do  in  a  no-can-do 
world,  a  story  of  how  a  Marine  officer  in  Iraq,  a  small  network-design  company  in  California,  a 
nonprofit  troop-support  group,  a  blogger  and  other  undeterrable  folk  designed  a  handheld 
insurgent-identification  device,  built  it,  shipped  it  and  deployed  it  in  Anbar  province.  They  did  this 
in  30  days,  from  Dec.  15  to  Jan.  15.  Compared  to  standard  operating  procedure  for  Iraq,  this  is 
a  nanosecond...  Before  fastening  our  seatbelts,  let's  check  the  status  quo.  As  a  high  Defense 
Department  official  told  the  Journal1  s  editorial  page,  "We're  trying  to  fight  a  major  war  with 
peacetime  procurement  rules."  The  department  knows  this  is  awful.  Indeed,  a  program  exists, 
the  Automated  Biometric  Identification  System:  retina  scans,  facial  matching  and  the  like.  The 
reality:  This  war  is  in  year  four,  and  the  troops  don't  have  it.  Beyond  Baghdad,  the  US  role  has 
become  less  about  killing  insurgents  than  arresting  the  worst  and  isolating  them  from  the 
population.  Obviously  it  would  help  to  have  an  electronic  database  of  who  the  bad  guys  are, 
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their  friends,  where  they  live,  tribal  affiliation — in  short,  the  insurgency's  networks...  The  Marine 
and  Army  officers  who  patrol  Iraq's  dangerous  places  know  they  need  an  identification  system 
similar  to  cops  back  home.  The  troops  now  write  down  suspects'  names  and  addresses.  Some, 
like  Marine  Maj.  Owen  West  in  Anbar,  have  created  their  own  spreadsheets  and  PowerPoint 
programs,  or  use  digital  cameras  to  input  the  details  of  suspected  insurgents.  But  no  Iraq-wide 
software  architecture  exists...  On  the  night  of  Jan.  20,  Maj.  West,  his  Marine  squad  and  the 
"jundi"  (Iraq  army  soldiers)  took  the  MV  1 00  and  laptop  on  patrol.  Their  term  of  endearment  for 
the  insurgents  is  "snakes."  So  of  course  the  MV  100  became  the  Snake  Eater.  The  next  day 
Maj.  West  e-mailed  the  US  team  digital  photos  of  Iraqi  soldiers  fingerprinting  suspects  with  the 
Snake  Eater.  "It's  one  night  old  and  the  town  is  abuzz,"  he  said.  "I  think  we  have  a  chance  to  tip 
this  city  over  now."  A  rumor  quickly  spread  that  the  Iraqi  army  was  implanting  GPS  chips  in 
insurgents'  thumbs...  Over  the  past  10  days,  Maj.  West  has  had  chance  encounters  with  two 
Marine  superiors — Maj.  Gen.  Richard  Zilmer,  who  commands  the  30,000  joint  forces  in  Anbar, 
and  Brig.  Gen.  Robert  Neller,  deputy  commanding  general  of  operations  in  Iraq.  He  showed 
them  the  mobile  ID  database  device...  I  asked  Gen.  Neller  by  e-mail  on  Tuesday  what  the  status 
of  these  technologies  is  now.  He  replied  that  they're  receiving  advanced  biometric  equipment, 
"like  the  device  being  employed  by  Maj.  West."  He  said  "in  the  near  future"  they  will  begin  to 
network  such  devices  to  share  databases  more  broadly.  Bottom  line:  “The  requirement  for 
networking  our  biometric  capability  is  a  priority  of  this  organization."  As  he  departs,  Maj.  West 
reflected  on  winning  at  street  level:  "We're  fixated  on  the  enemy,  but  the  enemy  is  fixated  on  the 
people.  They  know  which  families  are  apostates,  which  houses  are  safe  for  the  night,  which 
boys  are  vulnerable  to  corruption  or  kidnapping.  The  enemy's  population  collection  effort  far 
outstrips  ours.  The  Snake  Eater  will  change  that,  and  fast."  You  have  to  believe  he's  got  this 
right.  It  will  only  happen,  though,  if  someone  above  his  pay  grade  blows  away  the  killing 
habits  of  peacetime  procurement,  [comments  in  brackets,  italics  and  bold  added  by  Jones 
and  McCaffery.j  (Henninger,  2007,  p.  A14). 

Introduction 

In  previous  research  sponsored  under  the  Naval  Postgraduate  School  (NPS)  acquisition 
research  program,  we  have  argued  (2005)  that  there  are  mismatches  and  discontinuities 
between  the  acquisition  decision  process  and  the  Planning,  Programming,  Budgeting  and 
Execution  System  (PPBES).  We  identified  a  number  of  problems  associated  with  the 
misalignment  of  these  two  Department  of  Defense  (DoD)  resource-decision  systems.  To  reduce 
misalignment,  we  recommended  significant  business  process  reengineering  of  both  systems. 
We  are  pleased  to  observe  that  some  of  what  we  recommended  was  implemented  by  the 
former  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  (USD  AT&L) 
Kenneth  J.  Krieg  and  the  military  departments  and  services  (MILDEPS).  However,  in  May  2006, 
the  Under  Secretary  stated  that  while  some  successful  reengineering  had  been  done,  more  was 
needed.  In  December  2006,  the  USD  AT&L  noted  that  he  was  planning  to  do  more  of  this  within 
his  own  staff  and  within  the  decision  processes  he  controls. 

With  respect  to  further  changes  to  bring  better  alignment  between  the  acquisition 
decision  process  and  PPBES,  we  now  conclude  that  not  much  more  is  likely  to  occur  soon 
despite  the  need  for  resolution  of  the  many  mismatches  between  the  two  processes.  The 
initiative  to  further  reform  PPBES  has  disappeared  with  the  departure  of  former  Secretary  of 
Defense  Donald  Rumsfeld  as  champion  of  administrative  transformation,  and  the  absence  of 
funding  to  finance  it  due  to  the  continued  demands  placed  on  the  DoD  to  finance  OIF  and  the 
Long  War.  Without  additional  PPBES  and  budget  reform,  we  do  not  believe  it  is  possible  to 
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improve  the  fit  between  resource  allocation  and  acquisition  decision  processes  in  the  near  term 
to  any  significant  degree. 

In  2006,  we  presented  our  research  at  the  NPS  Acquisition  Symposium  that  argued  for 
implementation  of  capital  budgeting  in  the  DoD  and  across  the  federal  government,  with 
emphasis  on  mirroring  to  some  extent  how  it  is  done  in  the  private  sector.  We  also  explained 
how  capital  budgeting  could  be  implemented  within  the  DoD  without  changing  the  congressional 
decision  process,  and  explained  some  of  the  issues  to  be  resolved  to  do  so.  We  are  pleased  to 
observe  that  the  office  of  the  USD  AT&L  has  been  implementing  some  of  our  recommendations 
on  capital  budgeting. 

For  the  2006-2007  acquisition  funding  cycle,  we  have  concentrated  on  four  areas  that 
we  report  upon  in  this  paper  for  the  2007  NPS  Acquisition  Symposium.  Our  guiding  assumption 
in  organization  of  our  research  and  this  report  is  that  to  understand  defense  acquisition  process 
reform,  it  is  necessary  to  know  something  about  the  organizational  and  managerial  context  into 
which  such  reform  must  take  place.  Accordingly,  we  provide  analysis  of  the  Enterprise 
organization  and  management  initiative  now  underway  in  the  DoD,  and  show  how  it 
encompasses  the  new  approach  to  defense  capabilities  thinking,  planning  and  management  as 
a  preamble  to  our  argument  for  acquisition  system  and  process  reform.  Along  the  pathway  to 
presentation  of  our  acquisition  reform  proposals,  we  show  how  the  development  of  new 
information  technology  should  enable  managerial  reform  of  the  type  we  recommend  in  a  way 
that  adds  value  to  DoD  acquisition  and  resource  management.  We  advance  our  analysis  in  part 
using  the  example  of  the  DoD  Global  Information  Grid  (GIG)  to  demonstrate  both  possibilities 
and  obstacles  to  be  overcome  in  the  application  of  IT  and  the  principles  of  netcentricity.  We  also 
assert  the  necessity  for  understanding  something  about  the  new  economics  of  organizations  as 
a  critical  intellectual  component  of  support  for  the  arguments  we  make  for  defense  acquisition 
and  resource-management  reform. 

The  first  part  of  this  report  provides  an  assessment  of  the  efforts  currently  in  progress  to 
apply  enterprise  management  in  the  Navy  and  the  DoD.  In  this  report,  for  purposes  of 
description  and  analysis  of  the  Enterprise  organizational  framework  and  structure  under  which 
change  has  been  partially  implemented  in  the  DoD,  we  use  the  example  of  the  Navy  and  the 
Naval  NETWAR  FORCENET  Enterprise  or  NNFE. 

In  the  second  part  of  this  report,  we  examine  the  introduction  of  capabilities-based 
thinking,  planning  and  decision-making  into  the  enterprise  organization  and  management 
systems,  and  into  the  analysis  required  to  support  defense  acquisition  planning  and  decision¬ 
making.  We  identify  and  analyze  some  of  the  issues  faced  in  application  of  capabilities-based 
planning  and  resourcing,  especially  those  relating  to  definition  of  capabilities  and  deriving 
methods  to  crosswalk  from  (a)  traditional  program-based  proposal  and  acquisition  management 
to  (b)  capabilities-based  proposal  and  management.  As  we  indicate,  implementation  of 
capabilities-based  planning  and  management  in  the  defense  acquisition  process  changes  and 
adds  to  the  tasks  to  be  performed  and  the  information  needed  for  decision  and  execution  by  the 
Office  of  the  Secretary  of  Defense  (OSD) — particularly  the  US  AT&L  and  the  Secretary 
(SECDEF),  the  Joint  Chiefs  of  Staff  (JCS)  and  the  military  departments  and  services 
(MILDEPS):  e.g.,  data  needed  to  build  the  POM,  the  SECDEF  budget  proposal  and  to  perform 
medium-term  capital  asset  acquisition  planning  and  decision-making  at  various  levels  within  the 
DoD. 


The  third  part  of  this  report  explains  the  role  of  new  technology,  including  information 
technology  (IT),  in  a  new  approach  to  defense  acquisition  and  budgeting.  We  argue  here  that  IT 
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provides  the  basis  and  potential  for  almost  all  managerial  and  systems  reform  in  the  DoD, 
basing  some  of  our  conclusions  on  evidence  from  theory  and  private-sector  practice.  More 
broadly,  to  understand  how  defense  acquisition  can  and  should  be  done  incorporating  new 
technology,  we  explain  the  vital  role  of  IT  in  moving  from  bureaucracy  to  hyperarchy  and 
netcentric  organization  to  add  value  in  reform  of  defense  acquisition  and  resource  management, 
i.e.,  to  enable  the  types  of  change  we  advocate  subsequently  in  the  report.  This  part  of  the 
report  also  advances  the  relevance  of  the  new  economics  of  organization  as  a  component  part 
of  the  theoretical  and  practical  underpinnings  for  reform  of  defense  acquisition  and  resource 
management.  In  essence,  we  argue  the  necessity  for  relying  on  markets  and  the  private  sector 
in  moving  from  bureaucracy  to  hyperarchy  and  netcentric  organization. 

The  fourth  and  final  part  of  the  report  outlines  and  articulates  our  proposals  for 
fundamental  reform  of  the  entire  defense  acquisition  management  system  and  decision 
process,  based  on  and  integrated  with  the  DoD  Enterprise  organization  and  management 
initiative  and  capabilities-based  analysis,  decision-making  and  implementation.  We  have 
developed  two  approaches  to  reform.  The  first  is  a  marginal  adjustment  set  of  changes  to  the 
current  system  to  make  it  more  efficient  and  productive  (i.e.,  to  reduce  cycle-time,  reduce  costs 
and  improve  quality  per  investment  dollar)  through  business  process  reengineering,  enterprise 
management  and  improved  use  of  information  technology  (IT). 

The  second  proposal  is  for  a  much  more  comprehensive  and  radical  reform  of  how  the 
DoD  acquires  and  procures  weapons  platforms  and  systems.  The  comprehensive  reform 
proposal  we  refer  to  as,  "privatization  and  marketization  of  DoD  acquisition."  In  essence,  this 
proposal  argues  that  to  operate  defense  acquisition  more  like  a  business,  using  better  business 
methods  and  processes,  it  is  necessary,  literally,  to  make  DoD  acquisition  a  business:  i.e.,  to 
move  much  of  the  present  DoD  process  out  of  the  government  and  to  firms  in  the  private  sector. 
In  addition,  we  argue  that  in  facing  the  make-or-buy  decision  as  all  corporate  entities  must, 
increased  and  better  acquisition  and  procurement  of  commercial  off-the-shelf  (COTS)  weapons, 
systems  and  equipment  from  a  worldwide  market  will  get  the  US  taxpayer  greater  "bang  for  the 
buck"  in  acquiring  weaponry  for  defense  and  will  better  and  more  quickly  meet  warfighter  needs. 

We  argue  that  the  key  advantage  of  the  more  comprehensive  reform  approach  is  to  take 
full  advantage  of  the  competitive  dynamics  of  an  international  defense  capital  asset  market  in 
the  same  way  that  large  firms  in  the  private  sector  operate  presently — rather  than  relying  on  the 
system  and  process  the  DoD  uses  now  which  is,  in  essence,  a  gigantic,  disconnected  and 
inherently  ineffective  government  bureaucracy  that  resembles  in  form  a  Cold  War-era,  Soviet- 
style,  long-range  planning  hierarchy  in  which  the  process  has  become  the  product.  We  argue  for 
a  transition  to  a  system  in  which  the  product  is  the  focus  of  decision  effort,  and  we  outline  how 
such  a  system  would  operate  and  some  of  the  most  important  issues  to  be  resolved  in 
privatizing  DoD  weapons  systems  acquisition. 


The  complex  nature  and  consequent  length  of  this  report  made  it  impossible  to 
reproduce  the  entirety  in  these  Proceedings.  To  read  the  rest  of  this  paper  in  full  text,  please 
see  www.acquisitionresearch.org. 
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Abstract 

Business  management  reform  efforts  have  been  part  of  the  US  Defense  Department 
agenda  for  decades.  Current  reform  efforts  have  explicitly  established  the  goal  of  generating, 
harvesting,  and  reinvesting  savings  from  business  management  reform  to  buy  more  capital 
items;  that  is,  they  have  focused  on  a  measurable  reallocation  from  operating  and  support  costs 
to  investment  within  a  given  budget  top-line.  While  this  would  seem  to  be  good  news  for  the 
acquisition  community,  recent  increases  in  the  defense  top-line,  largely  related  to  the  war  on 
terrorism,  have  not  necessarily  resulted  in  greater  allocations  for  acquisition.  An  examination  of 
the  factors  affecting  the  top-line  suggests  that  near-term  budget  uncertainty  is  likely.  An 
examination  of  current  and  past  defense  management  reforms  suggests  that  efficiency-seeking 
business  management  reforms  are  not  likely  to  generate  sufficient  resources  to  cover  a  budget 
decline  or  finance  significant  capital  reinvestment.  Instead,  management  reform,  including  on¬ 
going  reform  of  acquisition  management,  should  be  sustained  for  reasons  of  stewardship  and 
accountability. 

Keywords:  Defense  management,  defense  budgets,  management  reform 
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Introduction 

After  six  years  of  increasing  budgets  and  an  active  defense  management  reform  agenda 
in  the  G.W.  Bush  Administration,  it  now  appears  that  defense  budgets  may  be  headed  for  a 
more  uncertain  near-term  future,  and  the  future  role  of  management  reform  initiatives  may  be 
equally  uncertain.  Despite  significant  increases  in  defense  spending,  resource  allocation 
decisions  are  favoring  operating  expenses  over  capital  investments.  External  pressures  from 
the  budget  deficit,  rising  costs  for  entitlement  programs,  pressure  from  other  discretionary 
programs  in  the  federal  budget  and  public  opinion  regarding  defense  activities  suggest  that  total 
resources  allocated  for  defense  will  be  constrained  in  the  foreseeable  future.  Internal  pressures 
from  rising  costs  for  manpower,  health  care,  operations  and  maintenance  are  constraining 
discretionary  spending  inside  the  defense  budget — even  as  acquisition  costs  are  also 
increasing.  The  Department  of  Defense  (DoD)  and  the  individual  service  components  have 
been  looking  toward  major  management  reform  initiatives  as  a  means  to  save  and  reinvest 
resources  within  current  budget  projections.  What  are  the  implications  of  this  budgetary  and 
management  environment  for  acquisition? 

Why  the  Near-term  DoD  Budget  May  Be  Uncertain 

When  one  asks  whether  the  DoD  budget  top-line  will  rise  or  fall,  there  are  both  historical 
patterns  which  should  be  considered  as  well  as  internal  and  external  factors  that  affect  those 
patterns. 

Figure  1  displays  three  measures  of  the  defense  top-line  over  the  period  1940-2011.  In 
absolute  terms,  since  WWII,  defense  outlays  appear  as  a  cyclical  pattern  with  a  lower  limit  of 
about  $350  billion  and  an  upper  limit  of  about  $550  billion  in  FY2006  dollars.  The  cycles  are 
fairly  consistent  with  a  wavelength  ranging  from  15  to  21  years.  Fiscal  Year  2007  is  18  years 
since  the  last  peak  in  1989,  suggesting  that  if  the  long-term  pattern  repeats,  defense  will  face 
declining  absolute  top-lines  beginning  within  the  next  few  years  and  continuing  for  7-10  years. 
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Defense  Spending  1940-2011 

Source:  Historical  Tables,  Budget  of  the  United  States,  2007 


Figure  1.  Defense  Top-line  1940-2011:  Various  Measures 

(OMB,  2007) 

In  relative  terms,  since  WWII,  defense  spending  as  a  percentage  of  GDP  has  fallen 
rather  steadily  from  a  peak  of  14.2  percent  in  1952  to  a  low  of  3  percent  in  1999-2000.  In  2006, 
defense  spending  stood  at  about  4.1  percent  of  GDP.  Similarly,  defense  spending  as  a 
percentage  of  federal  outlays  has  also  fallen  significantly,  from  a  peak  of  69.5  percent  in  1954  to 
a  low  of  16.1  percent  in  1999.  In  2006,  defense  was  19.8  percent  of  federal  outlays.  Today, 
approximately  1/5  of  federal  government  spending  is  for  national  defense,  and  4/5  are  for  other 
functions  of  government;  in  the  1950s,  2/3  of  government  spending  was  defense.  Should  the 
long-term  trends  continue,  defense  can  expect  declining  relative  top-lines. 

Simply  identifying  the  trend  of  the  last  50  years  does  not  mean  that  trend  will  apply  to  the 
next  3-5  years.  We  do  not  have  the  knowledge  to  make  a  point  prediction.  More  than  a  long¬ 
term  trend  is  needed,  and  there  are  other  factors  that  may  affect  the  top-line — those  both 
external  and  internal  to  the  department.  Apart  from  the  obvious  effects  of  the  war,  certain 
external  factors  are  associated  with  federal  fiscal  policy  and  political  dynamics. 

External  Fiscal  Factors 

Deficits.  As  of  2006,  the  US  has  experienced  four  consecutive  years  of  budget  deficits 
following  four  consecutive  years  of  budget  surpluses.  The  Bush  Administration’s  position  is  that 
it  will  eliminate  such  annual  deficits  by  2012.  When  an  administration  or  congress  wishes  to 
reduce  a  deficit,  generally  it  requires  a  combination  of  increasing  revenues  and  decreasing 
outlays.  If  defense  maintains  even  a  steady  proportion  of  federal  spending  during  an  overall 
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decline,  it  will  lose  top-line.2  But  the  correlation  between  defense  spending  and  deficits  may  be 
even  more  closely  connected. 


Defense  Spending  vs  Budget  Deficit  (Surplus),  1979-2006 

in  constant  2006  dollars 

Correlation  coefficient  =  -0.64,  significant  at  0.001 
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Figure  2.  Defense  Spending  and  Deficits,  1979-2006 

(OMB,  2007) 

Figure  2  plots  budget  deficits  against  defense  spending  from  1979  to  2006.  It  is  readily 
apparent  that  as  defense  spending  rose,  deficits  increased;  as  spending  on  defense  fell,  deficits 
fell  to  the  point  of  achieving  surpluses.  Defense  spending  does  not  necessarily  cause  deficits; 
rather,  each  could  be  affected  by  a  third  factor  such  as  an  unstable  international  situation. 
However,  the  apparent  relationship  suggests  near-term  uncertainty  for  the  defense  budget  as 
deficit-reduction  policies  take  hold. 

Mandatory  Programs.  Defense  spending  has  fallen  from  2/5  of  federal  spending  to  1/5 
in  the  past  40  years,  supplanted  by  mandatory  spending;  see  Figure  4.  From  1966  to  2006, 
interest  on  the  debt  and  “all  other  spending”  accounted  for  roughly  the  same  proportion  of 
federal  spending — about  41  percent.  The  remaining  59  percent  was  largely  defense  spending  in 
1966;  at  that  time,  defense  spending  was  three  times  more  than  Social  Security  spending.  In  the 
intervening  40  years,  the  proportion  of  federal  spending  on  defense  was  reduced  more  than  50 
percent  while  Social  Security  rose  by  40  percent.  Health  programs,  new  in  1966,  represent 
nearly  as  much  as  defense  in  2006. 3 


2  Of  course,  if  growth  is  kept  below  the  rate  of  inflation,  there  could  be  deficit  reduction  with  a  nominal 

increase  in  the  top-line. 

3  If  one  allocates  the  health-care  related  programs  from  defense  and  “all  other  spending”  and  adds  them 
to  the  Medicare  &  Medicaid  wedge,  the  total  health  care  portion  grows  to  about  25  percent  while  defense 
falls  to  about  19  percent.  See  table  16.1  of  the  Historical  Tables  that  accompany  the  federal  budget  for 
Fiscal  Year  2008. 
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Figure  3.  Components  of  Federal  Spending 

(OMB,  2007) 

The  Congressional  Budget  Office  estimates  that  this  shift  will  continue.  It  estimates  that 
in  2016,  the  mandatory  programs  will  cost  $1 ,274  billion  more  than  in  2006 — rising  to  a  total  of 
$2,793  billion  (CBO,  2006).  That  is  more  than  all  the  projected  federal  outlays,  mandatory  and 
discretionary  spending  in  2007.  The  CBO  projects  that  in  the  period  from  2008-2017, 
discretionary  budget  authority  will  increase  at  2.0  percent  per  annum,  relative  to  GDP  growth  of 
4.5  percent,  while  mandatory  spending  will  grow  at  5.9  percent  (CBO,  2007). 

Non-defense  Discretionary  Spending.  Defense  top-lines  not  only  feel  pressure  from 
growing  entitlement  programs,  but  may  also  face  pressure  from  other  discretionary  programs. 
From  1985  to  2006,  total  discretionary  outlays  as  a  percentage  of  GDP  fell  from  10.0  percent  to 
7.8  percent.  Of  those  2.2  percentage  points,  defense  spending  accounted  for  2.1  and  other 
discretionary  spending  accounted  for  0.1  (CBO,  2007).  Nearly  the  entire  reduction  in 
discretionary  spending  was  absorbed  by  defense.  The  last  few  years  of  that  history  tell  a 
different  story,  however.  From  2001  to  2006,  total  discretionary  outlays  as  a  percentage  of  GDP 
rose  from  6.3  percent  to  7.8  percent.  Of  that  1 .5  percentage  point  increase,  defense  accounted 
for  1 .0  point  and  non-defense  accounted  for  0.5  (CBO,  2007).  Figure  4  displays  a  40-year  trend 
in  discretionary  spending  separating  defense  from  non-defense  spending.  We  see  again  the 
cyclical  nature  of  defense  spending  with  a  counter-cyclical,  but  steadily  upward  trend  in  non¬ 
defense  spending. 
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Figure  4.  Defense  &  Non-defense  Spending  by  Administration 

(OMB,  2007) 


External  Political  Factors 

How  responsive  are  Washington  decision-makers  to  the  demands  of  the  public?  The 
evidence  strongly  suggests  that  if  the  general  public  believes  defense  spending  is  too  high, 
defense  spending  declines  and  vice-versa.  Studies  conducted  at  the  end  of  the  Cold  War 
demonstrated  empirically  that  the  desires  of  the  public  are  reflected  in  future  spending  decisions 
(Hartley  &  Russett,  1992;  Higgs  &  Kilduff,  1993). 

Updating  that  research,  Figure  5  displays  the  direction  and  strength  of  public  opinion 
about  defense  spending  with  subsequent  changes  in  that  spending.  Change  in  defense 
spending  correlates  strongly  with  the  direction  and  strength  of  public  opinion.  When  public 
opinion  favors  increased  defense  spending,  spending  has  tended  to  go  up  the  following  year; 
when  the  public  favors  a  decrease,  spending  tends  to  drop  the  following  year.  The  intensity  of 
public  opinion  also  forecasts  the  significance  of  the  gain  or  drop.  In  combination  with  the  data  in 
Figure  1 ,  one  might  conclude  that  the  public  has  a  “comfort  zone”  of  appropriate  defense 
spending  that  ranges  between  $350  billion  and  $550  billion  (FY2006  dollars).  Yet,  defense 
spending  for  FY08  is  projected  to  be  $593  billion  (FY2006  dollars). 
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Democratization  of  Defense  Spending,  1973-2006 


Direction  and  strength  of  public  opinion  vs  change  in  defense  outlays  one  year  later 
correlation  coefficient  =  0.57,  significant  at  0.001 
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Sources:  Smith  (2007)  and  OMB  (2007) 


Figure  5.  Public  Opinion  and  Defense  Spending 

(Authors,  derived  from  Smith,  2007;  OMB,  2007) 

What  does  the  public  currently  believe  about  defense  spending?  The  polling  data  used 
in  Figure  5  indicates  that  in  2006,  39.4  percent  of  Americans  favored  reduced  defense 
spending,  contrasted  with  26.8  percent  who  favored  an  increase  (Smith,  2007).  The  Program  on 
International  Policy  Attitudes  (PI PA)  at  the  University  of  Maryland  researches  American 
attitudes  regarding  international  and  foreign  policy  issues.  An  extensive  survey  in  2005  asked  a 
representative  sample  of  Americans  to  reallocate  a  hypothetical  $1000  in  income  taxes  across 
federal  programs  in  the  proportion  they  believed  was  most  appropriate.  Before  the  reallocation, 
they  were  shown  the  actual  2006  budget.  These  survey  respondents  would  have  cut  the  DoD 
baseline  budget  by  31  percent  and  war  supplemental  appropriations  by  35  percent.  They  would 
have  reallocated  most  of  those  resources  toward  reducing  the  deficit  and  improving  education. 
They  favored  increases  in  international  affairs  programs  and  would  have  reallocated  defense 
spending  away  from  strategic  and  heavy  capacity  (e.g,  nuclear  weapons,  bomber  aircraft  and 
capital  ships)  toward  manpower,  communication,  intelligence  and  special  operations  capabilities 
(PIPA,  2005). 

To  summarize,  there  has  been  fiscal  pressure  on  the  defense  top-line  from  growth  in 
mandatory  spending  accounts  and  non-defense  discretionary  accounts.  We  are  again  at  a  time 
when  deficits  are  a  point  of  concern.  Politically,  the  current  Administration  faces  increased 
congressional  opposition  to  military  policies;  public  support  of  defense  spending  is  waning. 

Time  will  tell  how  that  manifests  in  spending  decisions.  External  factors  are  vital  when 
considering  current  defense  management  reform,  but  internal  factors  may  be  more  important. 
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Internal  Factors 

Internal  factors  affecting  defense  spending  are  those  defense  leaders  are  more  able  to 
directly  control.  They  include  budgeting  and  spending  for  acquisition,  personnel,  and  operation 
and  maintenance.  The  policy  of  budgeting  for  the  war  separately  also  confounds  analysis  of 
defense  spending.  Before  looking  at  specific  factors,  let  us  take  a  broad  view.  From  the  low 
point  of  the  1990s  “procurement  holiday”  to  today,  the  DoN’s  budget  has  increased  46  percent 
in  real  terms,  but  the  size  of  the  fleet  fell  from  354  battleforce  ships  to  280;  aircraft  inventory  fell 
from  2,559  to  2,330;  and  the  number  of  personnel  (uniformed  and  civilian)  fell  from  929,358  to 
829,531  (Navy,  2007b).  While  spending  increased  by  half,  the  naval  forces  are  15  percent 
smaller.  The  last  time  defense  experienced  twelve  continuous  years  of  budget  growth  was 
1979-1990  during  which  time  the  fleet  grew  from  530  to  587  ships. 

To  sustain  the  current  goal  of  a  313-ship  Navy,  there  needs  to  be  sufficient  shipbuilding 
budget  authority  to  consistently  build  an  average  of  eleven  ships  a  year.  Since  1998,  Navy  5- 
year  budgets  have  planned  to  build  eight  ships  per  year  but  have  succeeded  in  building  only  six 
ships  per  year.  To  get  to  twelve  will  require  nearly  doubling  the  annual  shipbuilding  budget, 
currently  at  $11  billion  (CRS,  2006b).  Can  management  reform  efforts  generate  $10  billion  from 
the  other  accounts? 

Acquisition  Costs 

The  Department’s  appetite  for  major  acquisition  programs  and  the  cost  performance  of 
those  programs  continue  to  be  important  issues.  The  Joint  Strike  Fighter,  F-22A,  the  Army 
Future  Combat  System,  the  Air  Force’s  Transformational  Satellite  System,  the  Navy’s  DD-1000 
and  Virginia-class  submarine  programs  have  all  experienced  significant  cost  growth,  quantity 
reductions  or  schedule  slips  (CRS,  2005).  In  early  2007,  the  Navy  acquisition  executive  issued  a 
stop-work  order  for  the  relatively  affordable  Littoral  Combat  ship  when  the  first  ship  in  the  class 
experienced  costs  50  percent  over  budget.  Thus,  one  comes  to  the  difficult  realization  that  just 
as  other  internal  and  external  factors  are  putting  pressure  on  the  fiscal  resources  available  for 
recapitalization,  acquisition  itself  is  a  source  of  fiscal  stress. 

Personnel  Costs 

Pay  and  benefits  for  personnel  have  increased  in  recent  years.  At  the  same  time, 
accrual  accounting  changes  have  illuminated  some  costs  (such  as  the  accrued  costs  of  retiree 
heath  care)  which  have  always  been  there,  but  weren’t  explicitly  recognized.  The  activation  of 
tens  of  thousands  of  reservists  and  guardsmen  for  the  Iraq  war  resulted  in  higher  pay  and 
increased  long-term  liabilities  because  their  benefits  packages  have  been  expanded.  Figure  7 
plots  Department  of  the  Navy  spending  from  1997-2008  on  pay,  allowances  and  family  housing 
against  the  total  number  of  active  and  reserve  Navy  and  Marines.  In  constant  dollars,  the  cost 
per  troop  has  risen  47  percent.  In  an  effort  to  hold  outlays  constant,  the  policy  has  been  to 
reduce  the  size  of  the  force. 
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Figure  7.  Rising  Personnel  Costs,  Measures 

(Navy,  2007) 

Operations  and  Maintenance  Costs 

Looking  back,  O&M  spending  in  the  Department  of  the  Navy  since  1997  (including  the 
significant  Marine  Corps  participation  in  GWOT)  has  remained  a  steady  32  percent  of  DoN 
spending,  but  that  is  at  a  time  when  the  force  structure  has  decreased  15  percent  (Navy, 

2007b).  Looking  forward,  the  war’s  toll  on  equipment  will  drive  up  O&M  requirements  for  the 
next  few  years,  particularly  in  depot  maintenance  (GAO,  2006).  However,  that  is  not  readily 
apparent  in  the  budget.  The  FY2008/2009  defense  budget,  like  all  defense  budgets  since  9/11, 
was  tallied  and  submitted  separately  from  the  wartime  supplemental  budget.  Because  of  that 
practice,  the  baseline  budget  for  Marine  Corps  depot  maintenance  shows  a  decrease  from  $372 
million  in  2006  to  $71  million  in  2008  (Navy,  2007a).  Separately,  the  Marine  Corps  requested 
an  FY2008  supplemental  appropriation  of  $490.6  million  for  this  account.  If  one  has  the 
wherewithal  to  marry  the  two,  they  will  see  that  depot  maintenance  is  actually  projected  to 
increase  from  $372  million  to  $562  million,  not  decrease.  Program  and  budget  analysts,  in  an 
attempt  to  separate  a  “peacetime”  budget  from  the  cost  of  war,  now  produce  twice  the  number 
of  documents  (leading  to  twice  the  legislation  and  twice  the  number  of  accounts  to  manage) — 
none  of  which  individually  accurately  portrays  the  true  level  of  activity. 

Other  supplemental  appropriation  issues 

The  wartime  supplemental  appropriations  confound  analysis  of  defense  spending 
further.  When  the  Navy  submitted  its  FY2008  budget,  it  announced  cuts  in  the  acquisition  of 
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aircraft  (Castelli,  2007)  but  replaced  a  large  number  of  those  aircraft  in  the  supplemental 
requests.  For  example,  nine  H-1  helicopters  were  cut  from  the  base,  but  requested  in  the 
supplemental;  eight  H-60  helicopters  were  cut  from  the  base,  but  nine  were  requested  in  the 
supplemental;  four  V-22  tilt-rotor  aircraft  were  cut  from  the  base,  and  three  were  requested  in 
the  supplemental  (despite  the  fact  the  V-22  has  not  been  used  in  the  war)  (Navy,  2007). 
Theses  practices  cause  distortions  in  baseline  budgets  that  may  negatively  affect  future  DoD 
budgets  after  the  use  of  supplemental  appropriations  ceases  unless  some  type  of  “recapture” 
takes  place. 


Summary 

Will  the  defense  top-line  rise  or  fall  over  the  next  few  years?  Long-term  historical  trends 
suggest  that  by  all  measures  (%GDP,  %Outlays,  constant  dollars),  defense  spending  is  likely  to 
fall.  If  one  takes  into  consideration  factors  in  the  near-term,  we  see  evidence  of  downward 
pressure.  External  factors  such  as  growing  entitlement  programs  and  growing  discretionary 
programs  restrict  the  room  for  growth  in  the  defense  budget,  particularly  during  periods  when 
there  is  political  pressure  to  reduce  deficits.  In  addition,  the  newly-elected  Democratic 
leadership  in  Congress  and  the  direction  and  strength  of  public  opinion  question  increased 
defense  spending.  Internal  factors  push  for  a  higher  top-line  but  include  significant  inflationary 
effects,  so  top-line  increases  have  bought  less  force  structure  (but  not  necessarily  less 
capability).  Supplemental  appropriations  for  the  war  are  likely  to  continue  and  provide  an 
opportunity  for  the  defense  department  to  seek  relief  from  budget  pressures.  The  Bush 
Administration  requested  a  significant  defense  budget  increase  in  FY2008,  11  percent  more 
than  in  FY2007;  but  the  DoD  is  already  exhibiting  signs  of  fiscal  stress.  No  one  can  say  for 
certain  what  will  happen  in  the  future,  but  it  appears  the  defense  budget  is  under  both  increased 
stress  as  well  as  increased  scrutiny. 

Defense  Business  Management  Reform 

How  does  this  budget  situation  relate  to  business  management  reform  within  the  DoD? 

A  review  of  the  recent  history  of  defense  business  management  reform  and  an  examination  of 
the  current  management  reform  agenda  suggest  some  implications  for  both  budgeting  and 
managing  in  a  time  of  uncertain  budgets. 

Recent  History  of  Defense  Business  Management  Reform 

The  George  W.  Bush  Administration  also  came  to  office  with  a  management  focus.  The 
President’s  Management  Agenda  (PMA)  addresses  five  areas  targeted  for  management  reform 
throughout  the  federal  government:  human  capital,  improved  financial  management,  competitive 
sourcing,  electronic  government,  and  budget  and  performance  integration  (Rumsfeld,  2003). 

In  the  DoD,  an  initial  management  objective  was  to  “increase  effectiveness  through 
increased  accountability  and  efficiency”  (Rumsfeld,  2002),  with  emphasis  on  cost  reduction, 
improving  quality,  reducing  redundancies,  and  adopting  best  business  practices.  Secretary 
Rumsfeld  emphasized  the  creation  and  use  of  metrics  to  quantify  performance  improvements, 
and  he  sought  to  focus  the  Department’s  resources  on  core  functions  (Francis  &  Walther,  2006). 
To  improve  its  efficiency,  the  DoD  is  tackling  several  significant  challenges:  cost  reduction, 
organizational  realignment  and  cultural  issues  (Walker,  2004). 

The  DoD  has  identified  six  major,  strategic,  high-leverage  initiatives  called  Business 
Enterprise  Priorities  (BEPs).  These  BEPs  include  achieving  better  visibility  into  personnel, 
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acquisitions,  materiel,  finances,  common  supplier  engagement  and  real  property  accountability 
(DoD,  2006a).  Oversight  of  defense  business  transformation  is  conducted  by  the  Defense 
Business  Board,  an  advisory  panel  consisting  of  private-sector  executives  chartered  to  provide, 
“independent  advice  and  recommendations  on  effective  strategies  for  the  implementation  of 
best  business  practices  of  interest  to  the  Department  of  Defense”  (DoD,  2007). The 
Department’s  Business  Management  Modernization  Program  (BMMP)  and  Defense  Business 
Transformation  Agency  (DBTA)  were  created  to  institutionalize  parts  of  the  DoD  change 
program.  DBTA’s  strategic  objectives  include  improving  financial  stewardship,  enabling  rapid 
access  to  information,  and  reducing  the  costs  of  defense  business  operations  (Pair,  2007). 
DBTA  has  established  seven  directorates  to  manage  its  reform  agenda — including  the  Defense 
Business  Systems  Acquisition  Executive. 

The  Army,  Navy  and  Air  Force  have  developed  service-specific  management  reform 
agendas,  as  well.  Lean  Six  Sigma  is  the  Army’s  tool  of  choice  to  drive  across-the-board 
elimination  of  unnecessary  or  wasteful  processes,  the  reduction  of  process  variability,  and  the 
improvement  of  quality  (Army,  2006).  The  Air  Force  intends  to  create  new  processes  through 
its  Smart  Ops  21  program,  which  encompasses  the  tools  of  Lean,  Six  Sigma  and  Theory  of 
Constraints  (Lopez,  2005).  The  Navy’s  Sea  Enterprise  initiative  aims  to  reduce  costs  in  order  to 
provide  internally  generated  resources  for  reinvestment  and  recapitalization  (Clark,  2002).  Sea 
Enterprise  has  identified  three  imperatives:  change  the  culture,  improve  processes  and 
structures,  and  harvest  savings.4  The  Navy  is  also  undertaking  a  challenging  enterprise 
realignment  to  create  a  more  matrixed  organization  and  to  create  a  culture  of  enterprise-wide 
thinking. 

The  current  management  reform  agendas  have  a  defining  distinction:  they  focus  on 
generating  internal  savings  through  effective  cost  management  to  support  investment  and 
recapitalization  for  the  operating  forces.  This  vision  has  been  much  more  explicit  than  those  of 
past  reform  efforts.  It  provides  a  measurable  objective,  and  it  aligns  with  the  core  values  and 
mission  of  the  organization. 

There  are  practical  issues  associated  with  this  objective,  of  course,  such  as  how  to  re¬ 
allocate  operating  funds  to  investment  accounts  effectively  in  an  execution  year,  how  to  track 
and  apply  savings  within  the  current  accounting  systems,  and  how  to  distinguish  between  “real” 
savings  and  future  cost  avoidance.  Even  if  these  obstacles  can  be  overcome,  there  are  larger 
issues  that  threaten  to  undermine  the  management  reform  agenda  and,  in  turn,  reduce  or 
eliminate  recapitalization  funds  derived  from  management  reform. 

Management  Reform  at  Risk? 

The  current  defense  management  reform  agenda  is  driven  by  three  factors:  (a)  to  align 
changed  business  practices  with  force  transformation;  (b)  to  generate  resources  internally  to 
support  recapitalization  and  investment  in  future  combat  systems;  and  (c)  to  reduce  internal 
pressures  on  the  defense  budget  as  well  as  to  respond  to  external  pressures  on  the  defense 
budget.  If  these  imperatives  are  as  strong  as  they  appear,  what  then  could  undermine  the 
current  reform  effort  and  make  it  unsustainable?  There  are  four  clear  possibilities. 


4  For  a  more  thorough  description  of  current  reform  efforts,  see  (Dawe  &  Jones,  2005)  and  (DoD,  2006b). 
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1.  War  Costs.  If  operations  in  Iraq  and  Afghanistan  persist,  costs  associated  with  the  base 
defense  budget  and  the  war  effort  will  continue  to  diverge — particularly  if  war  funding 
continues  to  be  provided  through  non-traditional  budget  “bridges”  and  supplemental 
appropriations,  a  pattern  unseen  in  the  budgeting  for  past  wars  (CRS,  2006a).  In  this 
case,  management  associated  with  the  base  budget  will  operate  in  an  increasingly 
constrained  environment,  while  unconstrained  spending  will  continue  to  be  associated 
with  the  war.  This  is  hardly  an  environment  that  will  drive  culture  change  and  cost 
management  throughout  the  organization.  Moreover,  these  conditions  invite  the 
migration  of  base  budget  functions  to  the  less  constrained  and  less  cost-managed  war 
budget — resulting  in  distortions  in  the  base  budget  that  will  affect  future  budget 
decisions. 

2.  Losing  the  Vision.  Alternatively,  Congress  could  demand  that  appropriations  for  the  war 
efforts  migrate  to  the  base  budget.  In  that  case,  it  is  likely  total  defense  resources  will 
erode  as  they  are  combined  and  become  more  transparent.  Thus  follows  the  second 
possibility:  losing  the  reform  vision  to  current-year  “budget  drills”  to  meet  unfunded 
needs  or  to  fill  budget  “wedges”  in  the  execution  year.  Successful  reforms  may  depend 
on  demonstrating  that  the  cost  reductions  and  management  reforms  have  generated  the 
desired  savings  and  that  the  savings  have  been  applied  to  the  stated  objective.  Failure 
to  sustain  or  account  for  the  results  can  lead  to  a  loss  of  credibility  for  senior  leaders 
(Roberto,  2005).  If  budget  reductions  cause  the  recaptialization  goals  to  disappear  in 
favor  of  simply  meeting  reduced  budget  targets,  much  of  the  motivation  for  reform  could 
be  lost. 

3.  Change  Fatigue.  “Repetitive  Change  Syndrome”  is  experienced  when  organizations  too 
frequently  adopt  change  initiatives.  Such  frequency  generates  chaos,  burnout,  and 
incapacity  to  make  further  change,  thus,  harming  daily  operations  (Abrahamson,  2004). 
Observers  warn  against  adopting  change  initiatives  too  frequently  (Abrahamson,  2004). 
Defense  management  reform  has  been  a  continuous  theme  for  over  twenty  years. 
Sustaining  transformation  could  now  depend  on  learning  more  about  how  the 
organization  perceives  the  change  agenda  through  analysis  of  communications  and 
feedback  and  by  examining  the  knowledge,  attitudes,  and  actions  of  the  receptors  of 
communications  about  transformation. 

4.  Leadership  Change.  Leaders  change  frequently  in  the  DoD  among  both  uniformed  and 
senior  civilian  leaders.  It  often  appears  that  management  reform  initiatives  do  not  survive 
leadership  transitions,  notwithstanding  the  success  or  failure  of  any  particular  reform 
initiative.  The  current  Administration  is  in  its  final  two  years;  sustaining  its  management 
agenda  beyond  the  election  horizons  may  not  be  a  high  priority  in  the  permanent  DoD 
bureaucracy.  A  new  management  agenda  will  replace  the  current  “transformation” 
agenda,  though  the  next  administration  will  almost  certainly  face  the  same  challenges  to 
find  more  efficient  defense  business-management  practices  and  seek  reinvestable  cost 
savings  from  these  efficiencies.  Learning  to  institutionalize  the  principles  and  processes 
of  management  reform  will  be  important  to  future  leadership  transitions.  We  recommend 
a  research  and  education  project  to  facilitate  the  sustainment  of  improvements  in 
defense  business  management  practices  through  the  expected  leadership  transition  of 
2008-2009. 

Management  Reform  in  a  Declining  Budget 

How  then  should  management  reform  be  considered,  if  budgets  are  likely  to  decline  and 
significant  organizational  risk  factors  threaten  sustainment  of  defense  business-management 
reform  initiatives? 
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Efficiency-seeking  management  reforms  will  be  insufficient  to  make  up  the  budget 
shortfall.  Claimed  cost  savings  can  be  initially  impressive.  For  instance,  the  Naval  Air  Systems 
Command  claims  FY  2007  savings-to-date  amounting  to  13.9  million — composed  of  permanent 
cost  reductions  ($6.1  million)  and  potential  cost  reductions  from  improved  practices  ($7.8 
million)  (Navy,  2007c).  Overall,  the  Navy’s  business  transformation  advocate  claims  combined 
savings  from  Sea  Enterprise  initiatives  of  $27.7  billion  from  FY  2003-2005  (McCarthy,  2006).  In 
both  cases,  some  of  the  savings  are  realized  in  current-year  operations;  more  are  the  result  of 
projected  cost  reductions  and  revised  future  spending  requirements.  Nevertheless,  these 
savings  are  not  likely  to  close  the  budget  gap  if  defense  budgets  decline  and  there  is  insufficient 
evidence  to  indicate  whether  any  of  these  savings  can  be  tracked  to  reinvestment  in  capital 
accounts.  Even  if  efficiency-seeking  savings  reduce  a  significant  portion  of  the  internally 
generated  fiscal  stress,  cost-reduction  measures  through  management  initiatives  will  not  likely 
close  the  entire  fiscal  gap  because  they  only  address  the  internal  sources  of  stress.  The 
external  sources  are  untouched  by  efficiency  savings.  Moreover,  cost  reductions  are  dispersed 
throughout  the  organization,  are  difficult  to  harvest,  and  are  likely  to  be  hoarded  by  middle 
managers  in  the  face  of  tight  resources.  Savings  in  current-year  budgets  are  likely  to  be 
redirected  to  under-funded  current  requirements. 

Good  management  does  not  attract  resources.  There  is  little  evidence  to  suggest  that 
good  management  is  rewarded  with  larger  budgets.  Resource-allocation  decisions  are  policy 
choices  among  competing  demands.  Good  management  reduces  the  demand  and  may  lead  to 
reduced  marginal  future  resource  bases.  Even  the  movement  toward  performance  budgeting  in 
the  federal  government  leaves  unclear  the  link  between  performance  and  resource  allocation 
(Miller,  Hildreth,  &  Rabin,  2001).  Budgets  simply  do  not  grow  because  an  agency  gets  a  clean 
audit  opinion  or  reengineers  business  practices. 

If  efficiency-seeking  business-management  reform  cannot  be  viewed  as  a  solution  to 
declining  defense  budgets,  how  should  the  DoD  view  the  rationale  for  sustaining  management 
reform  initiatives?  There  are  three  strong  imperatives  for  sustaining  an  effective  management 
reform  agenda: 

Continuous  efficiency-seeking  management  improvement  in  the  DoD  should  be 
understood  as  driven  by  stewardship  concerns  and  the  requirements  of  the  operating  forces. 
The  DoD  has  the  responsibility  to  the  President,  the  Congress  and  the  public  to  be  a  good 
steward  of  the  highly  material  portion  of  the  federal  budget  it  manages.  There  is  a  justified 
expectation  that  the  DoD  will  manage  its  resources  well.  The  DoD  also  has  a  responsibility  to 
support  efficiently  the  varied  requirements  of  the  operating  forces:  people,  systems,  weapons, 
materiel.  The  policies  and  processes  of  management  reform  must  align  with  operational 
requirements,  especially  as  force  transformation  increases  pressure  on  the  business  side  of  the 
department. 

Large  budget  reductions  can  only  be  met  by  truly  transformational  responses.  Despite 
the  rhetoric  of  “transformation,”  the  history  of  management  reform  in  the  DoD  has  been  a  model 
of  incremental  continuous  change.  A  large  decline  in  the  defense  top-line  can  only  be  met  by 
transformational  changes  that  take  functions  AND  costs  out  of  the  department  permanently. 
Major  productivity  improvements  through  consolidations,  divestiture  of  both  functions  as  well  as 
assets,  cancelled  programs,  and  sharply  reduced  personnel  would  have  to  be  achieved. 

The  acquisition  community  must  view  itself  as  a  participant  in  cost-focused  management 
reform  rather  than  as  a  beneficiary.  Rising  acquisition  costs  are  a  major  source  of  internal  fiscal 
stress  for  the  DoD.  It  is  unlikely  that  cost  reductions  in  other  internal  areas  (such  as  personnel 
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or  operations  and  maintenance)  can  possibly  generate  savings  that  will  match  the  current 
growth  in  capital  costs,  let  alone  provide  funds  for  additional  reinvestment.  The  acquisition 
community  must  focus  on  its  own  cost-reduction  initiatives  to  help  generate  the  funding  needed 
for  recapitalization. 

Thus,  the  matter  of  sustaining  management  reform  in  a  time  of  uncertain  budgets  can  be 
approached  as  two  questions,  not  one:  What  are  the  best  budget-policy  decisions  regarding 
resource  levels  and  allocation  considering  the  salient  internal  and  external  factors?  And  how 
much  and  what  type  of  management  reform  is  needed  to  meet  the  standards  of  stewardship 
and  to  support  the  requirements  of  the  operating  forces? 

Conclusion 

We  have  both  investigated  business  management  reform  in  the  Department  of  Defense 
as  well  as  examined  the  question  of  uncertain  defense  budgets.  It  is  evident  that  pressure  on 
the  defense  top-line  comes  from  both  external  sources  and  internal  factors.  We  have  argued 
that  efficiency-seeking  management  reforms  can  only  address  the  internal  pressures  on  the 
defense  top-line.  We  have  identified  specific  threats  to  the  sustainment  of  defense 
management  reform  in  a  time  of  declining  budgets  and  have  offered  strategic  propositions  to 
consider  when  addressing  management  reform  in  a  time  of  uncertain  budgets. 
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The  Cultural  Revolution 


Presenter:  Dean  T.  Kashiwagi,  is  a  professor  at  Arizona  State  University’s  Del  E  Web  School  of 
Construction  and  the  Director  of  the  Performance  Based  Studies  Research  (PBSRG)  since  1992. 
PBSRG  is  the  worldwide  leader  in  improving  construction  performance  and  efficiency.  Kashiwagi  has 
developed  a  “hands  off’  approach  to  managing  contractors/vendors  in  the  construction  industry  or  in  any 
industry.  His  concept  is  contrary  to  traditional  price-driven  construction  procurement.  It  forces  the 
contractor/vendor  to  be  accountable — in  other  words,  minimizes  risk  for  the  facility  owner.  The 
technology  has  been  tested  over  450  times  totally,  using  $521 M  in  construction  projects  with  a  98% 
success  rate  (on  time,  on  budget,  and  high  quality).  This  is  one  of  the  few  documented  processes  that 
brings  better  value  for  the  owner  and  maximizes  the  profit  of  the  contractor.  It  is  currently  being  tested  in 
other  professional  areas  outside  of  construction. 

Presenter:  Nathan  B.  N.  Chong,  Chief,  Facility  Life  Cycle  Management  Division,  US  Army  Medical,  has 
been  involved  with  over  $1  Billion  worth  of  Army  medical  and  research  renovation  and  new  construction 
projects  around  the  world  for  the  past  22  years.  He  has  a  MS  in  Environmental/Civil  Engineering  from 
George  Washington  University  and  a  BSCE  in  Civil  Engineering,  Purdue  University.  Nathan  is  a  license 
Professional  Engineer  in  the  State  of  Hawaii,  Project  Management  Professional  (PMP)  Certified,  and 
Acquisition  Level  II  Certified — Facilities  Engineering  with  the  US  Department  of  Defense.  He  facilitated 
the  implementation  of  Performance  Information  Procurement  System  (PIPS)  within  the  Army  Medical 
Facility  acquisition  system.  His  career  objective  is  to  improve  the  Army  medical  design  and  construction 
processes. 

Author:  Kenneth  Sullivan 

Author:  Marie  Sullivan 

Dean  Kashiwagi,  PhD,  PE 

Performance  Based  Studies  Research  Group 

Arizona  State  University 

PO  Box  870204 

Tempe,  A Z  85287-0204 

Phone:  (480)  965-4273 

E-mail:  Dean.Kashiwaqi@asu.edu 

Nathan  Chong 

US  Army  Medical  Command 

2050  Worth  Road,  Suite  22,  MCFA 

Ft  Sam  Houston,  TX  78234-6022 

Phone:  (210)221-7938 

E-mail:  nathan.chonq@us.army.mil 


Abstract 

The  culture  within  the  Federal  Government  Acquisitions  is  based  on  the  Federal 
Acquisition  Regulations  {FAR)  and  its  interpretation,  often  placing  organizations/agencies  in  the 
cultural  environment  of  the  price-based  environment.  In  the  healthcare  system,  clients  depend 
on  the  qualifications  and  expertise  of  the  design  and  construction  team  to  meet  their  specific 
needs  and  requirements.  The  hiring  criteria  of  these  experts  have  been  primarily  based  on  low 
bid  or  relationships,  and  have  continuously  resulted  in  poor  performance.  The  US  Army 
Medical  Command  (MEDCOM)  (contracting  approximately  $100M  in  medical  renovation  awards 
per  year)  partnered  with  the  Performance  Based  Studies  Research  Group  (PBSRG)  at  Arizona 
State  University  to  create  and  test  an  information  environment  to  assist  in  alleviating  some  of  its 
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cultural  inefficiencies.  The  developed  information  environment  minimized  the  flow  of 
information,  forced  the  contractors  to  concentrate  on  value  and  the  assumption  of  risk,  and 
stimulated  an  atmosphere  of  accountability.  Through  the  system,  the  client’s  internal 
bureaucratic  resistance  was  minimized;  and,  without  controlling  the  various 
contract/procurement  processes,  MEDCOM  leadership  has  gained  control  of  the  performance  of 
their  infrastructure  revitalization  program  by  implementing  a  cultural  environment  of  information. 

Keywords:  Federal  Acquisition  Regulations  {FAR),  US  Army  Medical  Command 
(MEDCOM),  information  environment,  cultural  inefficiencies 

Introduction 

The  Performance  Based  Studies  Research  Group  (PBSRG),  experts  in  the  area  of 
performance  information,  has  used  the  Construction  Industry  Structure  (CIS)  diagram  to  define 
and  compare  the  characteristics  of  the  high-competition  environments:  Price  Based  and  Value 
Based  (Figure  1).  The  best-value  environment  focuses  on  securing  the  best-value  vendor  for 
the  owner  and  on  transferring  all  project  risk  to  the  outsourced  expert.  It  considers  the  vendor’s 
past  performance,  ability  to  identify  and  minimize  risk,  preplanning  foresight,  and  project 
knowledge.  It  requires  the  contractors  to  use  their  expertise  to  complete  a  project  that  fulfills  the 
intent  of  the  owner,  and  minimize  controlled  project  risk  at  the  beginning  of  the  project.  It  forces 
accountability  between  all  parties,  and  benefits  vendors  with  foresight,  experience,  skill,  and 
efficiency.  It  provides  an  environment  that  maximizes  contractor  profit,  while  minimizing  owner 
resources. 

In  contrast,  the  price-based  environment  focuses  on  using  minimum  standards  to  define 
the  requirement  of  the  contract,  in  order  to  ensure  that  the  minimal  requirement  is  met.  Due  to 
its  concentration  on  price,  it  encourages  the  contractors  and  vendors  to  translate  the  minimum 
requirement  to  a  maximum  in  attempts  to  lower  the  quality  of  the  delivered  product  to  gain  the 
competitive  advantage.  This  penalizes  the  high  performers  who  can  see  the  prospective  project 
from  a  visionary  view,  from  beginning  to  end,  and  seeks  to  minimize  change  orders  by  pricing  in 
items  which  will  need  to  be  done  that  are  missing  from  the  specifications.  It  discourages  the 
use  of  expertise,  and  asks  the  contractor/vendor  to  price  only  what  is  written — ignoring  the 
owner’s  intent.  It  promotes  project  dependency  on  change  orders  when  the  unspecified  or  client 
“unexpected”  events  occur  during  project  execution.  The  price-based  environment  gives 
inexperienced  contractors  the  competitive  advantage  over  experienced  contractors,  thus  driving 
the  experienced  contractors  to  move  from  a  position  of  minimizing  risk  to  a  position  of  ignoring 
project  risk.  This  culture  results  in  the  contractor  relying  on  the  client  to  manage,  direct,  control, 
and  inspect,  and  become  reactive  instead  of  proactive.  This  trend,  which  is  clearly 
demonstrated  in  Figure  2,  has  the  following  ramifications: 

1 .  Penalizes  contractors  who  carefully  preplan,  understand  the  scope  of  the  project,  and 
price-out  the  project. 

2.  Motivates  contractors  to  take  the  low  price  at  the  last  minute,  not  knowing  whether  they 
are  meeting  the  specifications,  and  when  receiving  the  award,  further  price  shopping  to 
ensure  that  they  can  meet  the  specifications  for  the  lowest  possible  price. 

3.  Encourages  all  manufacturers  to  ensure  that  their  products  meet  only  the  lowest 
possible  quality  to  get  the  largest  possible  volume  sales  of  contractors  trying  to  get  the 
lowest  price. 
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4.  Promotes  the  bypass  of  education  and  personnel  training  in  the  industry,  leading  to  a 
critical  shortage  of  trained  personnel. 

5.  Leads  to  poor  construction  performance  (not  on  time,  not  on  budget,  and  not  meeting 
the  client’s  expectations). 


High 


III.  Neaotiated-Bid 

Qualified  vendors  invited 

Owner  selects  vendor 

Negotiates  with  vendor 

Vendor  performs 

II.  Value  Based  (Process) 

Best  Value  (Performance  and 
price  measurements) 

Quality  control 

Contractor  minimizes  risk 
(transfer) 
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Figure  1.  Construction  Industry  Structure  (CIS) 

(Kashiwagi,  2004) 
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Figure  2.  Impact  of  Price-based  Bidding:  Increase  of  Risk 

(Kashiwagi,  2004) 


Relative  Analysis  of  the  Two  Environments 

A  relative  analysis  of  the  two  environments  (price  based  and  best  value)  leads  to  the 
following  deductive  conclusions.  Performance  can  only  occur  when  risk  is  transferred  to  an 
entity  that  has  the  capabilities  of  minimizing  the  risk.  The  best-value  environment  ensures  high 
performance  by  transferring  risk  to  the  best-value  contractor  who  can  verify  past  performance, 
send  their  best  personnel,  identify  risk  that  they  control  and  do  not  control,  and  develop  a  plan 
to  minimize  the  risk  that  they  do  not  control.  The  owner’s  resulting  risk  in  this  environment 
(assuming  that  the  client  did  pick  the  best  value — highest  performer  for  the  lowest  price),  is  the 
interface  or  the  seam  between  the  client  and  the  contractor,  or  in  other  words  the  risk  that  the 
contractor  does  not  control  (Figure  3).  Alternatively,  the  price-based  environment  passes  risk  to 
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the  contractor  with  the  lowest  price  without  ensuring  that  they  can  minimize  the  risk,  hence:  the 
poor  performance  of  the  construction  industry. 

When  risk  is  transferred  to  a  party  unable  to  minimize  the  risk,  the  party  must  be 
managed,  inspected,  and  controlled.  In  consequence,  the  owner’s  risk  in  the  low-price 
environment  is  the  potential  that  the  minimally  trained,  managed,  and  directed  contractor/vendor 
may  not  do  what  they  are  directed  to  do  (Figure  3).  The  price-based  environment  has  reflected 
this,  in  a  heavy  overhead  for  transaction  costs  relating  to  management,  direction,  control, 
inspection,  and  communications  that  would  be  eliminated  if  the  client’s  process  were  more 
efficient.  This  has  also  translated  to  a  higher  requirement  of  people  needed  to  maintain  the 
system  (due  to  the  inefficiency  of  the  process).  Accordingly,  there  is  more  confusion  in  the 
price-based  environment  due  to  the  management,  decision-making,  unrealistic  expectations, 
attempts  to  control  others,  use  of  leverage  (making  a  party  do  free  work  or  work  that  they  are 
incapable  of  doing),  and  the  lack  of  performance  information  of  key  individuals,  contractors,  and 
the  client’s  personnel.  Without  simple,  easy-to-understand  measurements  that  consider  the 
vendor’s  capabilities,  the  price-based  environment  is  adversarial:  where  every  participant, 
regardless  of  whom  they  work  for,  protects  themselves  before  they  protect  the  company  they 
work  for,  or  the  client/user  for  whom  the  construction  is  being  built. 


Low  Bid 


Client 

Procurement  j  (  Designer  ) 

(OQ  Vendoj) 

(Subcontractor) 

Best  Value 


Figure  3.  Difference  in  Risk  Between  Price-based  and  Best-value  Environments 


The  above  characteristics  are  supported  by  documentation  of  construction  delivered  by 
the  price  based  environment  (AGC,  2005;  Butler,  2002;  CM,  2005;  Doree,  2004;  Fitz-Gibbon  et 
al. ,  2006;  Guo,  2006;  Markus,  1997;  NDU,  2005;  Post,  1998;  State  of  Hawaii,  2002).  The 
characteristics  identify  management  as  the  key  component  to  the  price-based  environment;  and 
leadership,  or  the  alignment  of  resources  that  can  minimize  risk  as  the  key  component  to  the 
best-value  environment.  The  understanding  of  the  Construction  Industry  Structure  (CIS)  and 
the  impact  of  the  price-based  bidding  identifies  the  following  leadership/management 
characteristics  as  well  as  the  projected  goal  of  personnel  who  are  trying  to  move  from  the  price- 
based  environment  (management  and  control)  to  the  best-value  environment  (leadership): 

Management 

1.  Focus  on  relationships 
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2.  Lack  of  performance  information  on  critical  elements  of  the  contractor’s  team  (relative 
ability  to  finish  on  time,  on  budget,  and  meet  the  client’s  expectations) 

3.  Direction/Control  by  client’s  personnel 

4.  Decision-making  performed  by  client’s  personnel  instead  of  contractor  personnel 

5.  Maximum  communication/documentation  passed  between  client  and  contractors 

6.  Duplication  of  cost  estimates  quantities  and  approvals 

7.  Passing  of  risk  without  regard  to  relative  ability  to  minimize  risk 

8.  The  most  important  person  in  the  process  becomes  the  client’s  procurement  agent — 
although  they  have  no  technical  expertise  and  take  no  responsibility  for  cost  or  time 
overruns 

Leadership 

1 .  Performance  information  used  to  minimize  decision-making 

2.  Transfer  of  risk  to  those  that  can  minimize  risk 

3.  Replacement  of  the  client  management,  direction,  and  inspection  with  contractor/vendor 
self-  documentation/regulation 

4.  Minimization  of  client  decision-making,  documentation,  and  flow  of  information 

5.  Process  installed  to  ensure  that  the  best-value  contractors/vendors  know  how  they  will 
minimize  risk 

6.  Contractors/vendors  address  performance  and  risk  in  terms  of  value  chain,  supply  chain, 
and  overall  transaction  costs 

Information  Measurement  Theory  and  Best  Value  Test  Results 

PBSRG  has  been  testing  best-value  procurement  using  an  information  based 
Performance  Information  Procurement  System  (PIPS).  PIPS  is  built  on  a  foundation  of 
principles  outlined  by  the  Information  Measurement  Theory  (IMT).  IMT  is  a  set  of  deductive 
logic  models  which  predict  future  results  based  on  relative  measurements.  The  following  are 
the  major  concepts  or  models  (Kashiwagi,  2004): 

1 .  Decision-making  requires  individuals  to  use  their  subjective  bias  and  experience  to  solve 
a  situation  in  which  they  have  insufficient  information  to  predict  an  outcome. 

2.  Decision-making  brings  risk. 

3.  Decision-making  is  minimized  when  the  decision-maker  has  accurate  information. 

4.  Dominant  information  is  information  that  will  minimize  the  need  for  decision-making. 

5.  It  is  difficult  for  one  organization/person  to  control  the  actions  of  another  individual. 

6.  People  and  organizations  are  predictable  with  enough  information. 

7.  Past  performance  and  future  capability  to  perform  on  unique  events  can  be  predicted. 

8.  Experienced  personnel  can  identify  future  activities  in  an  event  before  it  happens.  They 
can  identify  and  prioritize  risk,  and  they  will  have  a  plan  to  minimize  the  risk  before  it 
happens. 
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The  following  test  results  and  measurements  of  PIPS  have  validated  the  above 
concepts: 

1 .  Duration  of  testing:  1 3  years. 

2.  Research  Funding:  $6.2M 

3.  Number  of  tests:  480 

4.  Construction  volume/scope  of  tests:  $500M 

5.  Largest  projects:  $100M  City  of  Peoria  Wastewater  Treatment  DB  project  (2007)  and  the 
$53M  Olympic  Village/University  of  Utah  Housing  Project  (2001) 

6.  Performance  of  contractors  in  tests  (on  time,  on  budget  with  no  contractor-generated 
cost  change  orders,  meeting  client’s  expectations):  98% 

7.  Surprise  factor  of  nonperformance:  Less  than  1% 

8.  Management  effort  of  client’s  construction  managers:  minimized  by  80  to  90% 

(University  of  Hawaii  (2000)  and  University  of  Minnesota  (2006),  and  the  ability  of  project 
managers  to  deliver  10  times  the  amount  of  projects  (State  of  Hawaii  (1997-2001) 

9.  Awards:  2005  Corenet  Global  Innovation  of  the  Year  Award  for  testing  at  Harvard 
University,  and  the  2000  Tech  Pono  Award  for  the  testing  at  the  State  of  Hawaii 

10.  Clients  in  the  Public  Sector:  FAA,  US  Army  Medical  Command,  USCG,  States  of 
Washington,  Wyoming,  Utah,  Georgia,  Hawaii,  and  Missouri,  City  of  Peoria,  AZ,  City  of 
Miami  Beach,  and  Universities  (University  of  Hawaii,  Arizona  State  University  and 
University  of  Minnesota) 

11.  Clients  in  the  Private  Sector:  General  Dynamics,  Raytheon,  Schering  Plough,  United 
Airlines,  Motorola,  Honeywell,  IBM,  Boeing,  Intel,  and  International  Rectifier 

12.  Risks  and  Reason  for  Stopping  PIPS:  the  champion/expert  of  Best  Value/PIPS  moves  or 
retires,  political  change,  someone  in  the  organization  feels  threatened  and  stops 
process,  organization  is  too  inefficient,  ineffective,  and  bureaucratic  to  make  process 
work 

Theoretical  Concepts  of  Best  Value 

The  Best  Value/PIPS  process  (shown  in  Figure  4)  is  composed  of  three  primary  steps: 

1.  Selection  Phase  (Filter  1-4):  Identification  of  the  Best  Value 

2.  Preplanning  /Quality  Control  Phase  (Filter  5):  Forcing  the  best  value  to  preplan  and 
minimize  risk  that  they  control  and  do  not  control  through  a  PIPS  Quality  Control  Plan  or 
Risk  Plan  and  schedule 

3.  Risk  Management  Phase  (Filter  6):  Management  of  the  construction  project  through  risk 
minimization 

The  selection  phase  attempts  to  differentiate  the  performance  and  expertise  of 
competing  vendors.  This  is  done  through  the  collection  of  each  contractor’s  past  performance 
information  (from  key  individuals  as  well  as  the  general  contractor  and  critical  subcontractors), 
risk-assessment/value-added  plan,  and  interview  ratings.  It  is  important  to  note  that  if  the 
contractors  cannot  differentiate  themselves  through  their  past  performance,  identification  of 
project  risk  out  of  their  control,  plans  to  minimize  uncontrolled  risks,  value  added  options,  and 
interview,  there  is  nothing  wrong  with  awarding  the  project  based  on  the  best  price  (as  the 
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contractors  proved  they  are  all  the  same).  The  client  should  not  make  decisions  to  assist  any 
contractor  to  become  competitive.  No  contractor  should  be  assisted  by  being  given  a  second 
chance,  redoing  their  cost  estimate,  or  given  information  from  other  contractors  that  could 
possibly  make  them  more  competitive.  In  best  value,  every  contractor  is  competitive,  and  every 
contractor  has  a  chance  to  differentiate  themselves  without  biased  assistance  from  the  client’s 
representative. 
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Figure  4.  Best  Value  Natural  Selection 

(Kashiwagi,  2004) 

The  Best  Value/PIPS  process  forces  the  best-value  contractor  to  take  its  price,  risk- 
assessment/value-added  plan  and  interview  statements  into  the  second  phase  of 
Preplanning/Quality  Control.  In  the  preplanning/PIPS  Quality  Control  Phase,  the  contractor 
concentrates  on  minimizing  the  project  risks.  A  schedule  listing  the  major  milestones  in  the 
project  is  developed.  A  QC  plan  is  also  compiled  by  the  contractor  which  includes  a  list  of  risks 
out  of  the  contractor’s  immediate  control  in  conjunction  with  a  detailed  plan  to  minimize  each 
risk.  Technical  risks  are  not  included,  as  the  contractor  minimizes  risks  that  it  controls  by 
meeting  the  requirements  of  the  specification.  Only  after  the  owner  is  satisfied  with  the  pre¬ 
planning  performed  by  the  contractor  will  the  contract  be  awarded. 


Once  the  project  has  been  initialized,  the  contractor  enters  the  Risk  Management  Phase. 
Every  week  during  project  execution,  the  contractor  is  required  to  submit  a  Quality  Assurance 
(QA)  plan  and  Weekly  Report  to  the  owner.  The  QA  plan  is  a  checklist  of  the  risks  identified  in 
the  previous  phase  that  ensures  that  each  risk  is  being  monitored  and  minimized  according  to 
the  directives  included  in  the  QC  plan.  If  the  risk  cannot  be  minimized  according  to  client  pre¬ 
approved  QC  efforts,  the  risk  is  reported  on  the  weekly  report  along  with  unforeseen  risks 
adversely  impacting  the  schedule  or  budget.  The  client  is  then  obliged  to  pay  for  additional  time 
and  effort  (Figure  5).  This  process  transfers  the  risk  to  the  contractor,  who  then  uses  the 
mechanism  of  the  QC  plan  to  make  all  parties  accountable  thorough  communication, 
coordination,  and  preplanning. 
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Unforeseen  Risks 
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Figure  5.  Information  Environment 

The  selection  process  ensures  the  procurement  of  the  best-value  contractor  and  the 
transfer  of  all  project  risk  to  the  contractor.  The  QC  plan  and  the  weekly  risk  report  then  defend 
the  high-performing  contractor  by  identifying  risks  out  of  the  contractor’s  control  and  the 
contractor’s  limited  abilities  to  impact  the  risk.  Because  it  is  reviewed  by  the  client’s 
representative,  the  information  is  usually  very  accurate.  The  QC  plan,  QA  checklist,  weekly 
report,  and  schedule,  also  help  to  regulate  the  contractor’s  work.  All  the  elements  are 
incorporated  into  the  contractor’s  contract  on  award.  At  the  end  of  the  project,  the  contractor  is 
rated  by  the  owner,  and  the  rating  modifies  the  past  performance  rating  of  the  contractor  by 
50%  (Figure  6). 


Figure  6.  Best  Value  DMAIC  Closed  Loop 

(Kashiwagi,  2004) 

Information  System 

The  owner  is  able  to  compile  a  group  of  individual  contractor  weekly  reports 
(spreadsheets  submitted  to  the  client  weekly)  into  a  Director’s  Report  (Figure  7)  which  can  give 
a  Facility/Construction  director  valuable  measurements  of  risk/performance  for  the  organization 
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as  a  whole,  as  well  as  a  prioritization  of  the  risks.  The  Director’s  Report  also  allows  for  the 
comparison  of  contractors,  project  managers,  project  integrators,  inspectors  or  design 
professionals  involved  in  the  projects.  The  report  provides  accurate,  timely  risk/performance 
information  that  disables  bureaucracy  and  identifies  where  risk  is  being  created.  For  the  first 
time,  it  gives  a  director  a  simplistic  information  system  with  minimal  maintenance  that  deters 
nonperformance  by  highlighting  nonperformance  quickly  and  accurately. 


UNFORESEEN  (UF)  &  OWNER  (O)  RISK  I 

#  UF  Risks  affecting  budget'schedule 

0 

Days  Delayed  (Calendar) 

0 

Additional  Cost 

$ 

#  0.  Risks  affecting  budget'schedule 

1 

Days  Delayed  (Calendar) 

0 

Additional  Cost 

$  2,000.00 

"Above  risks  include  both  potential  and  resolved” 

CONTRACTOR  RISK 

#  of  Unaddressed/Overdue  Risks 

1 

#  Unrated  Risks 

0 

#  Risks  affecting  budget'schedule 

2 

Risk  to  Schedule  (calendar  days) 

15 

Risk  to  Cost 

$  1,000.00 

Contractor  Satisfaction  Rating 

10.0 

"Above  risks  include  both  potential  and  resolved  " 

OVERALL  CURRENT  STATUS  | 

Days  Delayed  (Calendar  days) 

5 

%  Delayed 

0.8% 

Contractor  %  Delayed 

0.8% 

Additional  Cost 

$  2,000.00 

%  Over  Award 

9.2% 

Contractor  %  Over  Award 

0.0% 

Project  Satisfaction  Rating 

10.0 

Risk  Number 

1.16 

solved  risks.” 

7.1  Individual  project  performance 


TOP  10  RISK  RANKING  PROJECTS 

No. 

Project 

Risk# 

1 

Design  for  Two  HVAC(s)  &  Windows 

52.94 

2 

Tpe  III  WP 

17.71 

3 

WP  Replace  C  Tower  HVAC  Sys 

13.61 

4 

Rpr/Rpl  Elec  Distr  Sys 

11.91 

5 

Renew  Troop  Medical  Clinic 

11.07 

6 

Fire  Damage  Repair 

10.51 

7 

Construct  Hyperbaric  Chamber 

10.50 

8 

Fault  Current  &  Over  Dev  CS 

10.35 

9 

Convert  Warehouse  to  Admin.  Area 

10.10 

10 

WP  Fire  Protection  Upgrade 

10,07 

7.3  Risk  ranking  of  projects 


DIVISION  OVERVIEW 

11/03/06 

Total  Awarded  Budget 

$172,751.944  23 

Current  Cost 

$174,484,794  47 

Over  Budget 

$1,732.850  24 

PROJECT  OVERVIEW 

Total  Number  of  Projects 

116 

%  Projects  On  Time 

85% 

#  of  Jobs  Delayed 

17 

%  Projects  On  Budget 

86% 

#  of  Jobs  Over  Awarded  Budget 

16 

#  of  Projects  Missing  Owner  Ratings 

0 

AVERAGE  PROJECT 

Project  Budget 

$  1  489  240  90 

#  of  Risks  per  Job 

314 

Owner  Generated  Risks 

1.66 

Number  of  overdue  nsks 

0.25 

%  Over  Awarded  Budget 

3.39% 

%  over  budget  due  to  owner 

3.23% 

#  of  Days  Delayed 

8.76 

#  of  days  delayed  due  to  owner 

7.62 

Owner  Rating 

9.17 

Risk  Number 

2.42 

7.2  Overall  organization  performance 


CONTRACTOR  OVERVIEW 

Contractor  A 

Contractor  B 

Contractor  C 

Total  Awarded  Budget 

S 15725  858 

S52  274  179 

$  2  885  433 

Currant  Cost 

$16  480  244 

$52  274  179 

S  2  885  433 

OVERVIEW  OF  PROJECTS 

Total  Number  of  Pro,ects 

25 

18 

6 

*  of  Jobs  Da  a/ad 

4 

2 

4 

w  of  Jobs  Ovar  Awardad  Budgat 

9 

0 

0 

AVERAGE  PROJECT 

%  Over  Awarded  Budget 

13.93% 

0.00% 

0,00% 

*  of  Da/s  Delayed 

684 

233 

9  50 

Owner  Rating 

8  03 

8  89 

10  00 

Risk  Number 

4.15 

2.94 

2.41 

7.4  Comparing  vendor  performance 


Figure  7.  Director’s  Report 


Implementation  of  Best  Value/PIPS  in  the  US  Medical  Command 
(MEDCOM) 

In  order  to  implement  Best  Value/PIPS  into  the  US  Medical  Command  (MEDCOM),  the 
system  was  adjusted  in  order  to  assure  compatibility  with  the  FAR  and  AFARS.  As  a  result,  the 
following  changes  were  made: 

1.  The  preplanning/quality  control  period  is  performed  during  a  preconstruction  period  after 
the  award  of  the  contract. 

2.  The  technical  and  non-technical  concerns  of  the  client/user  are  given  to  all  contractors. 
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The  Best  Value/PIPS  process  can  be  implemented  on  Design-Build,  CM@Risk,  Design- 
Bid-Build,  IDIQ  contracts,  or  on  designers.  The  US  Army  Medical  Command  has  implemented  it 
on  IDIQ  contracts,  and  directed  the  contractors  to  use  a  best  value  process  that  includes 
preplanning,  Quality  Control  that  minimizes  risk  that  the  contractor  does  not  control,  weekly  risk 
reporting,  and  client  identification  of  contractor  performance. 

Once  the  IDIQ  contractors  had  been  educated  in  the  Best  Value  process,  they  began  to 
realize  that  it  would  assist  them  to  be  more  efficient,  make  the  client’s/user’s  representatives 
accountable,  and  maximize  their  profits  without  charging  more  (win-win).  The  contractors 
began  to  realize  that  the  process  was  a  very  successful  enterprise  model  that  used  best 
business  practices,  motivated  their  personnel  to  improve,  and  measured  their  own  performance. 
The  majority  of  the  contractors  began  requesting  their  personal  performance  measurements. 

The  US  Army  Medical  Command  effort  has  grown  in  stages: 

1 .  First  stage:  MEDCOM  officials  supported  the  system,  instituted  training  sessions  for  their 
own  personnel,  and  introduced  the  contractors  to  the  process  (duration  of  1  year). 

2.  Second  stage:  MEDCOM  instituted  the  system  into  their  specifications,  making  it  a 
requirement  to  select  on  best  value,  preplan  the  minimization  of  risk,  implement  a 
contractor  generated  QC  plan,  and  measure  performance.  Five  out  of  a  potential  seven 
IDIQ  contractors  attended  the  annual  Best  Value  conference,  which  included  detailed 
training  of  the  system,  at  their  own  expense.  The  contractors  viewed  it  as  a  process 
required  when  working  with  MEDCOM  (duration  of  1  year). 

3.  Third  stage:  The  information  environment  and  Director’s  Report  were  instituted. 
Performance  information  was  returned  to  the  IDIQ  contractors  on  both  project 
performance  and  ongoing  risk  minimization.  The  contractors  noted  that  the  Best 
Value/PIPS  structure  allowed  them  to  perform,  differentiate  themselves  based  on  value, 
and  increase  efficiency.  Irresolvable  project  problems  which  previously  have  migrated  to 
the  director  of  the  MEDCOM,  have  now  been  minimized  to  problems  with  easily 
identifiable  solutions.  Four  of  the  seven  contractors  generated  their  own  training  and 
measurement  systems.  Six  of  the  seven  contractors  attended  the  annual  Best 
Value/PIPS  training  education  at  their  own  expense.  One  of  the  contractors  began  using 
their  performance  information  to  educate  other  owners  on  the  efficiency  and 
effectiveness  of  the  Best  Value/PIPS  structure.  Contractors  now  view  the  Best  Value 
process  as  a  process  to  measure  themselves,  minimize  their  risk,  and  improve  their 
companies  (duration  of  1  year). 

4.  Fourth  stage  (current  stage):  Continuous  education  of  the  concepts  of  supply  chain 
optimization,  preplanning,  quality  control,  and  risk  minimization  are  being  facilitated. 

Both  the  client’s  personnel  (MEDCOM  representatives,  the  Corps  of  Engineers  (COE) 
procurement  personnel,  and  users)  and  the  contractor’s  personnel  are  learning  how  to 
maximize  the  effectiveness  of  the  system  instead  of  using  them  as  routine  additional 
duties. 
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Federal  Acquisition  Regulation  (FAR) 

The  intent  of  the  Federal  Acquisition  Regulation  (FAR)  (2002)  is  to  bring  the  government 
the  “best  value.”  In  the  delivery  of  construction,  the  FAR  recommends  sealed  bids.  However,  in 
FAR  36.103b  and  6.04(b)(1),  it  states  that  if  the  use  of  sealed  bids  cannot  effectively  deliver  the 
best  value,  the  request  for  proposal  process  using  criteria  other  than  price  can  be  used.  There 
is  ample  evidence  that  implies  the  delivery  of  construction  cannot  be  treated  as  a  commodity. 

The  term  “best  value”  is  mentioned  34  times,  and  the  term  “low  bid”  is  mentioned  19 
times  in  the  FAR.  Best  Value  means:  the  “expected  outcome  of  an  acquisition  that,  in  the 
estimation  of  the  Government,  provides  the  greatest  possible  benefit  to  the  requirement”  ( FAR 
2.1).  The  benefits  include  (FAR  102.2(b)): 

1 .  Satisfy  the  customer  in  terms  of  cost,  quality,  and  timeliness  of  the  delivered  product  or 
service  by,  for  example — 

■  Maximizing  the  use  of  commercial  products  and  services; 

■  Using  contractors  who  have  a  track  record  of  successful  past  performance  or  who 
demonstrate  a  current  superior  ability  to  perform;  and 

■  Promoting  competition; 

2.  Minimize  administrative  operating  costs; 

3.  Conduct  business  with  integrity,  fairness,  and  openness 
FAR  1.102.2-1  (b)  states: 

(b)  Vision.  All  participants  in  the  System  are  responsible  for  making  acquisition 
decisions  that  deliver  the  best  value  product  or  service  to  the  customer.  Best  value  must  be 
viewed  from  a  broad  perspective  and  is  achieved  by  balancing  the  many  competing 
interests  in  the  System.  The  result  is  a  system  which  works  better  and  costs  less. 

FAR  1.102-2(b)(2)  states: 

(2)  The  System  must  provide  uniformity  where  it  contributes  to  efficiency  or  where 
fairness  or  predictability  is  essential.  The  System  should  also,  however,  encourage 
innovation,  and  local  adaptation  where  uniformity  is  not  essential. 

FAR  1.102-4  (e)  further  states  that: 

(e)  The  FAR  outlines  procurement  policies  and  procedures  that  are  used  by 
members  of  the  Acquisition  Team.  If  a  policy  or  procedure,  or  a  particular  strategy  or 
practice,  is  in  the  best  interest  of  the  Government  and  is  not  specifically  addressed  in  the 
FAR ,  nor  prohibited  by  law  (statute  or  case  law),  Executive  order  or  other  regulation, 
Government  members  of  the  Team  should  not  assume  it  is  prohibited.  Rather,  absence  of 
direction  should  be  interpreted  as  permitting  the  Team  to  innovate  and  use  sound  business 
judgment  that  is  otherwise  consistent  with  law  and  within  the  limits  of  their  authority. 
Contracting  officers  should  take  the  lead  in  encouraging  business  process  innovations  and 
ensuring  that  business  decisions  are  sound. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -456- 


The  FAR  addresses  the  use  of  the  PIPS  filters  of  past  performance  ( FAR  1 5.305(a)(2)), 
the  risk-assessment  plan/value-added  plan  ( FAR  15-305(a)(3)(i)),  and  the  interview  ( FAR 
15.102).  The  FAR  addresses  prioritization  of  alternatives  in  15.305  Proposal  evaluation: 

Proposal  evaluation  is  an  assessment  of  the  proposal  and  the  offeror’s  ability  to  perform 
the  prospective  contract  successfully.  An  agency  shall  evaluate  competitive  proposals 
and  then  assess  their  relative  qualities  solely  on  the  factors  and  subfactors  specified  in 
the  solicitation.  Evaluations  may  be  conducted  using  any  rating  method  or  combination 
of  methods,  including  color  or  adjectival  ratings,  numerical  weights  and  ordinal  rankings. 
The  relative  strengths,  deficiencies,  significant  weaknesses,  and  risks  supporting 
proposal  evaluation  shall  be  documented  in  the  contract  file. 

However,  the  Army  FARS  has  the  following: 

5115.304 — Evaluation  Factors  and  Significant  Subfactors 

(iv)  Must  be  qualitative.  Numerical  weighting  (i.e.,  assigning  points  or  percentages  to 
evaluation  factors  and  subfactors)  is  not  an  authorized  method  of  expressing  the  relative 
importance  of  these  factors  and  subfactors.  Evaluation  factors  and  subfactors  must  be 
definable  in  readily  understood  qualitative  terms  (i.e.,  adjectival,  colors,  or  other  indicators, 
but  not  numbers)  and  represent  the  key  areas  of  importance  to  be  considered  in  the  source 
selection  process.  The  direction  of  this  subparagraph  is  not  waivable,  either  on  an  individual 
or  class  basis,  as  an  AFARS  deviation. 

Therefore,  when  selecting  a  contractor,  AFARS  Best  Value  process  must  not  use  either 
weights  or  the  ratings  on  any  of  the  evaluation  factors  must  have  qualitative  ratings.  The  impact 
of  this  policy  is  the  lack  of  transparency  but  also  prevents  protests  due  to  the  inability  or  difficulty 
to  challenge  a  subjective,  nontransparent  system.  The  downside  to  this  type  of  system  is  that  it 
motivates  owner  representatives  to  make  decisions  instead  of  allowing  the  contractors  to 
determine  who  gets  the  project  based  on  a  preset  system  that  is  very  predictable.  However, 
Best  Value/PIPS  can  still  be  run  using  qualitative  ratings  on  past  performance,  risk 
assessment/value  added  submittal,  and  interviews. 

The  major  contribution  of  Best  Value/  PIPS  is  in  the  Preplanning/QC  phase  and  the  Risk 
Management  phase.  These  two  phases  can  be  written  into  the  IDIQ  specification.  The  phases 
provide  the  user  with  relevant  information  related  to  the  contractor’s  performance.  These 
components  are  not  a  procurement  issue,  but  a  client’s  requirement  to  ensure  wise  usage  of 
their  funding.  This  responsibility  must  be  fulfilled  and  is  periodically  checked  by  auditors  to 
ensure  the  funds  are  receiving  the  best  value  for  the  government. 


Resistance  to  Change 

The  movement  to  Best  Value  is  threatening  to  the  status  quo  due  to  the  following: 

1 .  The  contractor  becomes  the  expert,  the  center  of  the  universe,  the  most  important 
component  of  the  value  chain,  replacing  the  perch  the  procurement  officer  has  in  the 
low-bid  environment. 

2.  Best  Value/  PIPS  forces  the  government  to  release  control  to  the  outsourced  vendor.  It 
is  difficult  for  procurement  agents  to  release  control. 


n.-.i'.uMiAftr.  v  hsiu,, 
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3.  If  services  are  outsourced,  and  the  process  becomes  more  efficient  and  effective,  the 
government  will  need  less  personnel  to  make  it  work.  This  threatens  the  procurement 
community,  whose  members  may  feel  that  their  jobs  are  being  eradicated. 

4.  A  change  in  culture  brings  fear  to  the  government  community.  This  fear  is  exhibited  in 
many  ways:  resistance  to  change,  not  being  open  to  logic/best  practices,  using  the  FAR 
and  AFARS  and  interpreting  new  concepts  as  illegal,  increasing  transaction  costs  of 
other  participants  in  the  delivery  chain,  and  not  acting  in  the  best  interest  of  the 
government. 

Resistance  to  using  the  Best  Value/PIPS  process  has  included: 

1 .  Stating  that  Arizona  State  University  could  not  be  used  to  replace  the  COE  procurement 
process.  This  is  a  fear  that  is  totally  unjustified.  ASU  is  a  research/education  group. 
ASU  does  not  participate  in  the  procurement  selection. 

2.  Best  Value/PIPS  will  only  quantify  contractors  with  past  performance.  This  is  incorrect. 

If  a  contractor  has  no  past  performance  ratings,  the  client  is  instructed  to  rate  them  an 
average  rating.  This  rating  is  called  “I  don’t  know.”  The  contractor  could  be  a  very  high 
performer  based  on  the  owner’s  past  experience;  however,  the  client  should  not  bias  the 
system  by  making  a  decision  and  helping  a  contractor  get  in. 

3.  The  past  performance  is  only  good  past  performance.  In  order  to  win  a  true  best-value 
RFP,  the  contractor  must  send  their  best  people,  and,  therefore,  their  best  past 
performance.  In  this  case,  the  best  past  performance  is  indicative  in  the  way  the  best 
value  will  operate.  This  minimizes  client  decision-making,  forces  the  competition  among 
performers,  and  allows  the  transfer  of  risk  to  the  best  performer. 

4.  Risk-assessment  plans  are  general  and  ambiguous.  As  contractors  get  grounded  into 
the  system  of  identifying  the  risks  that  they  do  not  control,  the  risk  plans  will  get  better. 
However,  they  can  only  add  to  the  contractor’s  preplanning  and  thinking  and  do  not  act 
as  a  detriment. 

5.  Interviews  should  focus  on  technical  matters.  In  order  to  rate  contractors  on  technical 
matters,  the  government  has  to  be  the  expert,  make  decisions,  and,  therefore,  absorb 
the  risk.  This  system  has  proved  to  be  unsuccessful.  Contractors  who  can  answer 
concerns  simply,  in  non-technical  language,  are  high  performers.  Poor  performers 
cannot  simplify. 

6.  There  is  a  lack  of  competitive  range  and  discussions.  The  competitive  range  should  be 
identified  by  the  best  values. 

7.  The  scope  of  the  project  should  not  be  published  in  terms  of  budgets.  Best  Value/PIPS 
encourages  giving  the  budget  to  the  contractors  so  they  may  determine  risk.  However, 
giving  the  budget  is  not  mandatory.  Not  giving  the  budget  often  helps  the  low  bidders 
who  gain  the  advantage  with  low  price.  High  performers  always  minimize  risk,  and  if  any 
information  is  withheld,  the  risk  increases,  and  the  high  performers  increase  their  prices. 

8.  Sharing  of  offerors’  risk  plans  with  other  contractors  is  not  required. 
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Conclusion 

Best  Value/PIPS  is  a  cultural  revolution  for  the  government.  The  process/structure  has 
been  well  tested  over  an  extended  period  of  time.  Best  Value/PIPS  requires  the  procurement 
agents  to  release  control,  minimize  decision-making,  minimize  the  creation  of  transaction  costs, 
and  change  their  thinking.  Education  is  the  key.  The  government  needs  to  become  more 
efficient  and  effective. 
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Incentive  Contracts:  The  Attributes  that  Matter  Most  in  Driving 
Favorable  Outcomes 


Presenter:  Robert  L.  Tremaine,  is  an  Associate  Dean  for  Outreach  and  Performance  Support  at  the 
Defense  Acquisition  University’s  West  Region  campus  in  San  Diego,  CA.  He  is  an  Air  Force  Academy 
graduate  and  holds  a  Master’s  of  Science  in  Systems  Management  (Research  and  Development)  from 
the  Air  Force  Institute  of  Technology.  He  is  a  retired  Air  Force  colonel  with  over  25  years’  experience  in 
air,  missile,  and  space  weapon  systems  acquisition. 


Abstract 

Incentive  contracts  have  been  in  place  for  many  years.  They  represent  just  one  of 
many  contractual  tools  the  Department  of  Defense  has  at  its  disposal  to  drive  certain 
performance  behaviors.  Lately,  the  usefulness  of  incentive  contracts  has  come  into  question. 
The  dividends  have  not  been  readily  apparent.  This  research  study  set  out  to  determine  what 
generally  afforded  strong  correlations  between  incentive-type  contracts  and  expected 
performance  outcomes.  Twenty-five  weapon  system  acquisition  programs  offices  were 
interviewed  in  various  stages  of  their  acquisition  lifecycles.  A  standardized  questionnaire- 
survey  was  used  to  capture  the  data.  This  presentation  prepared  for  the  Fourth  Annual 
Acquisition  Research  Symposium  will  address  the  findings  and  include  a  few  key 
recommendations  intended  to  better  arm  the  acquisition  workforce  on  the  use  of  incentive 
contracts. 

Discussion 

In  the  past  several  years,  major  weapon  system  development  programs  have  drawn 
significant  attention.  The  reasons  are  varied.  In  some  cases,  costs  have  skyrocketed; 
schedules  have  experienced  significant  delays;  and  performance  levels  have  failed  to  meet 
government  expectations  despite  the  employment  of  management  tools  designed  to  control 
costs,  preserve  schedules  and  influence  performance  outcomes.  Some  of  these  management 
tools  (including  contractual  measures,  as  originally  conceived  and  specified  by  the  Federal 
Acquisition  Regulations  (FARs))  can  give  tremendous  flexibility  to  the  implementation  of 
government  contracts.  The  use  of  such  contractual  measures  is  one  of  many  handy  tools  in  a 
program  manager’s  toolkit  to  help  drive  performance  behavior.  However,  the  GAO  recently 
identified  an  apparent  disconnect  between  the  use  of  certain  measures  like  incentives  and 
expected  outcomes  in  weapon  system  acquisitions.  In  short,  it  appeared  that  incentives  were 
not  driving  performance  outcomes  as  originally  envisioned. 

So,  what  about  incentives?  Are  they  still  a  good  tool  to  drive  performance  behaviors 
despite  the  recent  criticism  and  doubt?  Have  organizations  found  a  way  to  effectively  apply 
incentives  and  demonstrate  the  usefulness  of  incentives?  The  answer  to  all  of  these  questions 
is,  “yes.”  There  is  no  “one  size  fits  all,”  but  the  incentive  attributes  that  seemed  to  matter  the 
most  in  influencing  performance  outcomes  for  the  25  programs,  and  generally  afforded  strong 
correlations  between  incentives  and  desired  performance  were  indicated  by  the  findings. 
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Conclusion 


Strongly  Communicated  Expectations  and  Feedback:  Frequent  and  unambiguous 
communication/feedback  made  a  noticeable  difference  for  incentive  contracts.  Even 
though  incentive  contracts  require  some  additional  administrative  burden,  the  outcome  justified 
the  increased  workload  of  feedback  for  most  programs  under  this  research  review.  Continuous 
and  open  dialogue  at  both  junior  and  senior  levels  led  to  early  discovery  and  timely 
reconciliation  of  many  known  issues  and  helped  keep  a  program  on  track. 

Metrics:  The  selection  of  key  and  enduring  measures  within  an  evaluation  period 
and  of  measures  that  could  be  connected  to  subsequent  evaluation  periods  made  a 
noticeable  difference  for  incentive  contracts.  Key  measures  can  validate  whether  or  not  a 
program  achieved  certain  necessary  intermediate  milestones  along  a  program’s  critical  glide 
path.  They  confirm  program  momentum.  They  served  as  an  early  warning  system — a  bell 
weather — and  answer  the  age  old  question,  “Are  we  on  track”?  They  also  fill  a  huge  role  as 
performance  benchmarks.  Those  interviewed  under  this  research  project  said  when  they 
effectively  employed  key  measures,  such  tools  also  helped  them  navigate  their  program 
pathway  despite  the  unavoidable  programmatic  turbulence.  Their  measures  surfaced  either  as 
two  types:  objective  and/or  subjective.  Without  question,  selecting  the  correct  type  of  measures 
presented  the  biggest  challenge.  The  ability  to  hard-wire  them  to  achievable  outcomes  makes 
objective  measures  like  Technical  Performance  Measures  (TPMs),  Cost  Performance  Indices 
(CPIs),  Schedule  Performance  Indices  (SPIs),  etc.,  invaluable  gauges.  They  served  as 
tremendous  forecasting  devices  when  they  were  carefully  connected  to  outcomes. 

Base  Fee:  The  incorporation  of  base  fee  in  award-fee  contracts  made  a  noticeable 
difference.  Of  the  25  organizational  interviews,  many  used  some  form  of  base  fee  on  cost- 
plus-award-fee  contracts.  Numerous  organizations  implementing  cost-plus-award-fee  (CPAF) 
valued  base  fees  as  a  leverage  tool.  Even  though  the  Federal  Acquisition  Supplement 
( DFARS )  216.405-2( c)(iii)  allows  up  to  3%  of  the  estimated  cost  of  the  contract  exclusive  of  fee, 
a  contractor  could  provide  “best  efforts”  for  the  award-fee  term  and  still  receive  no  award.  As  a 
result,  there  was  some  pressure  on  the  government  to  provide  a  portion  of  the  award  fee  for 
“best  efforts.”  Further,  our  research  team  found  that  senior  defense  industry  personnel 
welcomed  the  use  of  base  fee  to  better  delineate  the  difference  between  “best  efforts”  (e.g.,  fee) 
and  “excellence”  (e.g.,  award). 

Trained  and  Experienced  Personnel:  Training  and  experience  made  a  noticeable 
difference  for  incentive  contracts.  Nothing  seems  to  have  a  more  dramatic  impact  in  DoD 
like  training  and  experience.  Training  draws  it  roots  from  practical  experience.  It’s  systematic. 
We  learn  from  our  successes  and  failures  in  the  field  and  make  adjustments  accordingly  in  the 
way  we  train.  The  mantra,  “we  train  like  we  fight  and  fight  like  we  train”  is  pervasive  within  the 
warfighter  arena  and,  ultimately,  leads  to  advantages  on  the  battlefield.  Without  question, 
practical  experience  helps  build  better  training  programs.  It  can  overcome  unforeseen  shortfalls 
and  the  inevitable  prevailing  uncertainty  even  within  proven  systems.  It’s  no  different  for 
incentive-type  contracts.  Organizations  that  had  formalized  instruction  and/or  coached  their 
personnel  on  the  use  of  incentives  indicated  such  training  more  favorably  influenced  outcomes. 

The  exploitation  of  an  increasingly  popular  collaborative  medium  called  Communities  of 
Practices  (CoPs)  on  the  DAU  Acquisition  Community  Connection  offers  access  to  these 
particular  techniques  and  an  even  wider  array  of  current  experiences  and  lessons  learned 
regarding  incentives  ranging  from  the  general  to  the  specific.  The  DAU  has  already  established 
a  site  on  the  ACC,  Award  and  Incentive  Fee  Contracts.  See 
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(https://acc.dau. mil/CommunitvBrowser.aspx?id=1 05550).  These  and  other  collaborative 
training  aids  are  critical  because  once  an  incentive  strategy  is  in  place,  its  maximum  value  truly 
depends  on  its  ability  to  implement  techniques  that  drive  favorable  outcomes.  There’s  no  better 
source  of  experts  who  face  contract  incentive  challenges  every  day  than  the  acquisition 
workforce  who  is  charged  with  appropriately  implementing  the  techniques  that  drive  outcomes, 
appreciably. 
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Capabilities-centric  Acquisition:  A  System  of  Systems  View  of 
Acquisition  Management 

Presenter:  Colonel  Raymond  D.  Jones,  USA,  Aviation  Project  Manager,  Modular  Brigade 
Enhancements 

Colonel  Raymond  D.  Jones 
PEO  Ground  Combat  Systems 
(586)  574-8209 
Raymond.iones@us.army.mil 


While  heeding  the  profit  of  my  counsel,  avail  yourself  also  of  any  helpful  circumstances  over  and 
beyond  the  ordinary  rules.  -Sun  Tzu,  The  Art  of  War 

Introduction 

The  purpose  of  this  paper  is  to  begin  a  discussion  on  the  need  and  complexity  of 
managing  the  material  acquisition  process  from  a  capability  focused  perspective.  As 
warfighters  develop  doctrine,  tactics,  techniques,  and  procedures  for  the  current  and  future  fight, 
they  do  so  from  a  joint  and  combined  arms  perspective.  Battlespace  success  is  viewed  from  the 
combined  effects  of  multiple  systems  providing  a  synchronized  force  multiplier  for  the 
Commander.  Conversely,  our  acquisition  process  remains  trapped  in  a  historical  paradigm 
designed  to  meet  Cold  War  requirements.  This  paper  does  not  intend  on  offering  the  solution. 
This  is  merely  a  thought  piece  on  perspectives  from  someone  who  is  challenged  daily  with  the 
opportunity  of  developing  a  capability  management  process  for  integrating  future  capability  into 
the  current  force  organizational  construct. 

The  Department  of  Defense  is  challenged  with  balancing  weapon  system  modernization 
and  maintaining  an  operational  force  ready  to  fight  and  win  the  Global  War  on  Terrorism.  As  the 
Department  seeks  to  transform  itself  into  a  twenty-first  century  force,  the  acquisition  process  is 
stuck  in  a  Cold  War  mentality  focused  on  preserving  the  existing  platform-centric  approach  to 
acquisition.  Tomorrow’s  battlespace  will  be  a  network-centric  environment  derived  from  system- 
of-systems  within  which  the  sum  of  the  parts  generates  an  interdependent  capability  much  more 
effective  than  the  stand-alone,  platform-centric  environment  of  the  past.  Our  DoD  acquisition 
process  is  still  oriented  on  building  platforms  that  come  to  the  fight  as  applique  solutions,  rather 
than  seamlessly  integrated  warfighting  systems  designed  to  enhance  the  total  capability.  This 
dichotomy  is  straining  the  DoD  budget  by  focusing  our  limited  resources  on  an  ever-decreasing 
number  of  platforms  that  are  hugely  expensive  and  fall  short  of  meeting  the  ever-increasing 
number  of  capability  gaps  being  endured  by  our  warfighter. 

This  paper  will  examine  the  need  to  shape  the  DoD  Acquisition  Process  and  create  an 
acquisition  management  system  that  is  capability  centered  rather  than  platform  centered.  A 
knowledge-based  process  that  synchronizes  and  optimizes  capability  solutions  across  the  force 
is  needed  to  ensure  the  warfighter  needs  are  met  within  the  fiscal  constraints  of  the  budgetary 
process.  The  acquisition  process  needs  to  be  viewed  in  a  more  holistic  manner.  The  various 
aspects  of  acquisition  management  can  be  viewed  as  layers  in  a  system  of  systems  acquisition 
architecture.  A  Capability  Manager  must  be  established  to  address  all  the  layers  of  the 
acquisition  process — not  just  the  traditional  cost,  schedule  and  performance  metrics  typically 
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addressed  by  a  platform  Project  Manager.  The  layers  include:  Standards,  Requirements, 
Acquisition  Organization,  Material  Solution,  Material  Subsystem,  Operational  Organization,  and 
Battlespace. 


OPERATIONAL  ORGANIZATION 


ACQUISITION  ORGANIZATION 


Capabilities  Engine 


Figure  1.  Material  Acquisition  Process  Model,  “Capability  Engine” 

These  layers  are  interconnected  in  such  a  way  that  a  perturbation  on  any  layer  evokes  a 
response  in  all  layers.  By  addressing  capability  management  in  a  system-of-systems  way,  we 
can  better  align  our  management  processes  with  the  requirements  generation  process. 
Additionally,  by  inculcating  a  mindset  that  views  acquisition  management  in  a  system-of- 
systems  way  that  is  synchronized  with  the  capabilities-based  approach  to  developing  needs, 
one  is  better  able  to  determine  where  our  limited  resources  need  to  be  applied — mitigating  our 
current  management  approach  which  typically  over-resources  some  capability  at  the  expense  of 
other  warfighter  needs. 


Platform  Centric  Perspective 

As  the  Department  of  Defense  plans  to  invest  over  a  trillion  dollars  into  the  acquisition 
process  in  support  of  new,  more  capable  weapon  systems,  it  is  failing  to  address  the  root  cause 
that  has  resulted  in  ever-increasing  cost  over  runs:  performance  disappointments  and  program 
failures.  Since  1970,  the  percentage  of  cost  overrun  has  increased  to  as  much  as  40%  or  $15 
billion  dollars — even  though  there  have  been  as  many  as  eleven  policy  changes  attempting  to 
reform  the  DoD  acquisition  process.  According  to  the  GAO  report,  Major  Weapon  Systems 
Continue  to  Experience  Cost  and  Schedule  Problems  under  DoD  Revised  Policy  (2006,  April), 
the  following  table  illustrates  the  problem  facing  the  DoD. 
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Program 

Percent  Cost  Growth 

Percent  Schedule 
Growth  (months) 

Percent 

Development 

Remaining 

Aerial  Common 
Sensor 

45% 

24 

85% 

Joint  Strike  Fighter 

48% 

48 

78% 

Expeditionary  Fighting 
Vehicle 

30% 

23 

60% 

C-130  Avionics 
Modernization 
Program 

122% 

N/A 

N/A 

Global  Hawk 

166% 

N/A 

N/A 

Future  Combat 
System 

48% 

48 

78% 

Source:  GAO.  (2006,  April).  Major  weapon  systems  continue  to  experience  cost  and  schedule  problems  under  DoD  revised  policy. 
Washington,  DC:  author. 


As  we  invest  more  resources  into  fewer,  more  resource-intensive  programs,  we  need  to 
examine  the  root  cause  of  the  Acquisition  Process’s  failure  to  gain  control  of  its  cost,  schedule 
and  performance  challenges.  The  principle  reasons  often  cited  for  cost  overruns  are  the 
budgeting  and  requirements  process;  yet,  this  does  not  account  for  the  ever-increasing  dollars 
flowing  into  programs  with  less-than-impressive  results.  Additionally,  the  problem  is  not  a  lack 
of  available  policy  and  statutory  guidance.  Perhaps  we  are  organized  for  inefficiency  in  that 
program  offices  are  hardwired  to  solve  material  solution  challenges  from  a  singular  system 
perspective  and  are  not  trained,  organized,  or  resourced  to  leverage  opportunities  around  them 
in  other  program  offices  with  similar  challenges. 

Requirements 

The  battlespace,  which  our  material  solutions  are  supposed  to  support,  is  a  highly 
complex,  integrated  environment;  yet,  our  acquisition  system  continues  to  manage  programs 
from  a  singular  platform  perspective  without  considering  the  relationships  between  all  layers  of 
the  acquisition  process.  We  often  have  multiple  programs  born  from  a  single  requirement  that 
compete  for  the  same  scarce  resources,  even  though  these  programs  fundamentally  provide 
the  same  capability. 

We  assume  that  the  requirements  process  has  accounted  for  the  synchronization  of 
capabilities,  and  we  manage  our  programs  from  the  platform  perspective.  No  where  are  the 
total  force  requirements  mapped  against  all  the  program  of  record  material  solutions  to  inform 
the  process  from  a  capabilities  perspective.  Indirectly,  we  create  an  environment  that  does  not 
consider  the  whole  more  important  than  the  subordinate  parts.  In  an  environment  like  this,  it  is 
impossible  to  manage  from  a  capabilities  perspective,  since  each  of  the  Program  Managers 
considers  his/her  system  the  ideal  solution  for  the  relatively  ambiguous  requirements  being 
generated  out  of  the  DoD  requirements  process. 

Leadership 

To  address  the  volatile,  complex,  uncertain,  and  ambiguous  (VUCA)  acquisition 
environment,  the  services  choose  the  best  of  the  best  to  manage  our  programs.  We  provide 
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them  with  an  endless  set  of  policies,  statues,  and  directives  and  ask  them  to  develop  the  best 
warfighting  platform  money  can  buy.  Program  managers  are  selected  for  their  leadership  and 
management  skills  to  manage  specific  platforms  or  systems.  Our  process  of  selecting  Program 
Managers  reaches  into  our  organizations  and  looks  for  individuals  that  are  focused  on  getting 
the  mission  done.  When  we  give  them  the  mission  of  managing  a  specific  platform,  these  same 
managers  are  programmed  to  succeed  regardless  of  the  program  issues  to  their  left  and  right 
boundaries.  As  long  as  their  program  succeeds,  they  are  accomplishing  their  mission. 

Although  our  doctrine  emphasizes  system-of-system-level  integration,  we  organize  our  program 
offices  to  look  myopically  at  a  system  model  rather  than  at  system-of-systems. 

Funding 

Perhaps  the  most  challenging  aspect  to  this  problem  is  the  funding  allocation.  Budgets 
are  allocated  to  align  with  specific  material  solutions.  There  is  no  specified  Program  Element  in 
the  budgeting  process  that  recognizes  the  need  to  optimize  material  solutions.  Consequently, 
Program  Managers  report  their  successes  based  upon  a  budget  activity  that  views  cost  as  an 
independent  variable  against  schedule.  Funding  objectives  need  to  support  the  capabilities 
model — which  is  less  tangible  than  a  hardware-focused  model.  The  challenge  is  to  determine 
the  metrics  by  which  success  should  be  measured  and  how  to  place  a  dollar  value  on  that 
metric. 


In  order  to  address  this  disconnectedness,  we  must  change  the  very  nature  of  our 
organizational  processes  and  develop  an  acquisition  process  that  considers  the  whole  more 
important  that  the  individual  parts.  Additionally,  we  must  recognize  that  material  acquisition  is 
more  than  managing  cost  schedule  and  performance.  System  Program  Management  is  a 
subset  of  capability  management,  which  requires  managing  toward  a  collective  capability  and 
synchronizing  all  the  relevant  stakeholders  from  the  foundation  of  standards  to  battlespace 
integration.  It  includes  multiple  layers  that  need  to  be  synchronized  for  every  program  and 
balanced  across  the  needs  of  the  soldier  and  the  needs  of  the  nation  as  a  whole. 

Capability-centered  Acquisition 

Managing  to  a  capability  does  not  specify  any  unique  material  solution.  Capability 
management  strives  to  achieve  the  optimal  solution  across  the  organization — in  which  there 
may  be  many  material  solutions  for  a  unique  requirement  or  a  single  material  solution  for 
multiple  platforms.  The  objective  is  to  optimize  the  potential  solution  candidates  that  are 
designed  to  meet  a  requirement,  ensuring  the  best-value  solution  set  for  the  warfighter  and  the 
taxpayer. 

The  combined  effect  of  multiple  material  or  operational  solutions  might  provide  the 
relevant  capability  for  the  force  and,  subsequently,  for  the  individual  systems  within  the  force. 
Figure  2  shows  how  multiple  solutions  are  generated  from  a  single  requirement  and  blossom 
into  multiple  potential  solutions  based  upon  the  individual  Platform  Manager’s  perspective.  As 
the  specified  operational  requirement  is  translated  into  material  options,  Program  Managers 
each  begin  individual  solution  sets  or  material  solutions  with  regard  to  their  specific  program. 
From  the  material  solution  is  generated  an  expansive  set  of  subsystem  and  software  options.  In 
today’s  environment,  the  Project  Manager  spends  little  time  looking  toward  other  Project 
Managers  for  material  options,  resulting  in  a  plethora  of  programs  designed  to  meet  a  single 
requirement.  Ultimately,  the  Service  has  a  suboptimal  capability  solution  requiring  ever- 
increasing  time  and  money  to  support  the  individual  material  approaches  derived  in  each 
program. 
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Increase  Cost  Due  to  Multiple 
Solutions  for  the  Same  Requirement 


Non  Optimized  Platform  Centric 


Figure  2.  Acquisition  Process  Model  “Capability  Engine”  Reflecting  a  Non-optimized 

Material  Development  Strategy 

Conversely,  Figure  3  depicts  the  same  requirement  being  satisfied  in  a  more  optimized 
approach.  All  layers  of  what  I  refer  to  as  the  “Capabilities  Management  Engine”  are 
addressed  to  ensure  continuity  in  the  requirement  and  the  solution  set.  In  order  to  “qualify”  as 
an  acceptable  solution  for  a  specified  requirement,  all  layers  of  the  “engine”  must  be  considered 
and  linked.  The  path  through  the  engine  must  be  optimized  with  the  fewest  number  of 
subordinate  connections.  Minimizing  these  links  requires  a  Capabilities  Manager  with  the 
authority  to  influence  systems  horizontally  across  each  layer  of  the  engine  with  the  intent  on 
leveraging  solutions  sets  across  multiple  programs. 


Figure  3.  Acquisition  Process  Model  “Capability  Engine”  Showing  an  Optimized  Material 

Development  Strategy  Solution 


Capabilities  Engine: 

The  synchronizing  process  is  the  key  to  being  able  to  ensure  success  in  the  capabilities- 
centered  acquisition  environment.  The  “Capabilities  Engine”  needs  to  consider  all  aspects  of 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -468- 


the  requirement,  material,  organizational,  and  policy  environment  and  be  able  to  synchronize 
this  across  multiple  programs. 

•  Capability  Layer  1  (CL1):  Standards.  The  standards  layer  is  the  foundation  upon  which 
all  acquisition  programs  are  based.  Standards  encompass  policy,  statute,  regulations, 
treaties,  etc.  All  programs  must  be  tied  back  to  firm  standards  foundations  lest  they  be 
developed  in  contravention  to  the  authoritative  documents  upon  which  our  institutions 
are  formed.  This  may  seem  obvious,  but  it  is  a  critical  step  that  is  often  not  well 
understood  by  Program  Managers. 

•  Capability  Layer  2  (CL2):  Requirements.  The  requirements  process  is  the  beginning  of 
all  programs.  The  warfighter  determines  a  unique  need  and  validates  the  need  must  be 
satisfied  with  a  material  solution.  Simply  levying  a  requirement,  however,  is  insufficient. 
All  of  the  intended  and  unintended  impacts  of  requirements  must  be  completely  vetted 
with  regard  to  the  systems  that  are  already  fielded  and  those  in  the  development 
process.  The  Capability  Manager  will  link  the  requirement  to  the  appropriate  program 
offices,  the  material  solution  and  its  subsequent  subsystem  contributors  and  ultimately 
back  to  the  warfighter.  The  intent  is  to  establish  a  clear  path  with  an  optimized  solution 
from  requirement  inception  to  retirement. 

•  Capability  Layer  3  (CL3):  Acquisition  Organization.  This  is  simply  the  acquisition 
organization  that  is  most  appropriate  to  develop  a  specific  material  solution  for  a 
requirement.  Often,  a  single  requirement  will  be  levied  upon  multiple  program  offices. 
Without  a  Capability  Manager  to  synchronize  the  respective  program  offices,  the  risk 
remain  that  singular  solutions  will  be  developed  for  individual  platforms. 

•  Capability  Layer  4  (CL4):  Material  Solution.  As  material  solutions  are  developed,  the 
Capability  Manager  will  match  similar  solutions  and  optimize  the  set  to  minimize  the  total 
capital  outlay  necessary  to  meet  the  specific  requirement.  This  is  the  point  at  which  the 
Program  Managers  must  share  resources  in  order  to  minimize  the  total  cost  and  ensure 
a  synchronized  capability  is  maintained  to  meet  the  common  requirement.  Throughout 
this  synchronization  process,  however,  each  Program  Manager  must  develop  a  solution 
that  is  consistent  with  his/her  platform-strategic  plans.  This,  in  fact,  is  how  the  Program 
Manager  is  chartered  and  funded.  It  is  the  Capability  Manager’s  mission  to  ensure  the 
overall  solution  is  optimized  across  the  organizations. 

•  Capability  Layer  5  (CL5):  Material  Subsystem.  CL5  is  similar  to  CL4  with  the  exception 
that  the  total  number  of  potential  solutions  is  orders  of  magnitude  greater.  Here  is  where 
the  “good  idea”  cost  driver  is  most  apparent.  The  multitude  of  subcomponents  for  a 
system  are  ripe  picking;  any  vendor,  government  organization,  or  anyone  with  a  good 
idea  to  solve  an  engineering  problem  will  use  whatever  means  necessary  to  convince  a 
Program  Manager  to  resource  their  project.  Left  unchecked,  this  is  where  programs  run 
the  greatest  risk  of  being  desynchronized  with  each  other.  Additionally,  this  is  where 
programs  tend  to  allow  “requirements  creep”  and  contribute  to  increasing  program  costs. 
The  Capabilities  Manager  must  synchronize  subsystems  between  program  offices  and 
ensure  they  are  consistent  with  each  of  the  subordinate  layers.  Failure  in  this  area 
manifests  itself  as  inefficient,  redundant,  or  worse:  incompatible  systems  on  the 
battlefield. 

•  Capability  Layer  6  (CL6):  Operational  Organization.  The  operational  test  community 
and  the  user  must  look  at  the  combined  nature  of  the  material  solutions  and  assess 
whether  or  not  the  material  solution  is  simply  a  slightly  better  applique  of  what  already 
exists.  The  user  must  hold  the  acquisition  community  accountable  for  developing 
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system-of-system  solutions  that  are  mutually  supporting  and  as  common  between 
systems  as  possible. 

•  Capability  Layer  7  (CL7):  Battlespace.  The  battlespace  is  the  future  state  at  which  a 
particular  service  views  its  warfighting  posture.  All  systems  must  support  this  objective. 
Lack  of  a  clear  link  to  the  future  state  might  reveal  program  flaws  or  non  relevance  to  the 
objectives  of  the  National  Military  Strategy  and  subordinate  service  objectives. 

As  a  requirement  is  developed,  a  clear  link  from  CL1  through  CL7  must  be  established 
and  managed.  The  Capability  Manager  can  synchronize  the  seven  layers  for  specific  platform 
solutions  across  multiple  program  offices.  Platform  Program  Managers  manage  their  programs 
horizontally,  with  respect  to  time  and  program  milestones.  The  Capability  Manager  manages 
“vertically”  within  each  phase  of  the  spectrum  of  Programs  attempting  to  optimize  the  material 
solution  across  multiple  platforms  and  through  the  seven  layers  of  the  capability  engine. 

Knowledge-based  Decisions 

Managing  from  a  capability-centered  perspective  is  significantly  more  complex  than  the 
traditional  platform-focused  approach.  One  must  account  for  all  aspects  of  the  organization  and 
be  willing  to  accept  being  dependent  upon  the  success  of  other  organizations  and  managers. 
This  is  inherently  counterintuitive  for  the  “traditional”  Project  Manager,  in  that  the  “success  at  all 
cost”  approach  to  doing  business  is  significantly  dependent  on  others  outside  of  the  Program 
Manager’s  sphere  of  influence.  Consequently,  managing  toward  an  optimized  capability  is 
highly  dependent  on  having  a  well-established  knowledge-based  process,  a  clearly  defined 
execution  strategy,  and  well-defined  roles  and  responsibilities.  Additionally,  leadership  at  least 
two  levels  up  must  emphasize  and  support  an  organizational  structure  that  facilitates  the 
seamless  execution  of  the  capabilities  model. 

A  knowledge-based  capability  model  will  enable  developers  to  be  reasonably  certain  at 
critical  junctures,  or  “knowledge  points”  in  the  acquisition  lifecycle,  that  their  products  are  more 
likely  to  meet  the  cost,  schedule,  and  performance  baselines  of  the  individual  contributions  of 
each  project  office  and  provide  an  optimized  solution  set  which  meets  both  the  warfighter 
requirement  and  mitigates  redundancy  and  excessive  resource  demand. 

Knowledge-based  capabilities  management  requires  that  someone  manages  the 
knowledge  process  and  has  the  authority  to  influence  the  platform-centered  processes.  As  the 
operational  commander’s  weakest  point  on  the  battlefield  is  at  the  seams  (the  boundaries 
between  units),  the  acquisition  Project  Manager’s  weakest  point  is  the  relationship  between  his 
program  and  that  of  another  Project  manager.  This  is  the  where  the  Capabilities  Manager, 
using  a  knowledge-based  process  will  strengthen  the  overall  capability  solution.  Figure  4 
represents  key  points  at  which  the  Capabilities  Manager  should  influence  the  processes  of  the 
platform-centered  Project  Manager. 
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Milestone  C 


•  Formal  Review  of  Capability  Engine 
at  each  Knowledge  Point  is  required 


Figure  4.  Knowledge  Points  at  Which  Capability  Manager  Validates  Optimization  of 

Material  Solution  for  a  Specified  Program 

The  Capabilities  Manager  will  synchronize  all  seven  layers  of  the  “Capabilities  Engine”  at 
each  of  the  three  knowledge  points  of  the  Platform  Project  Manager.  The  platform-centered 
Project  Manager  manages  across  time  with  regard  to  the  five  phases  of  the  acquisition  lifecycle. 
Those  phases  include  the:  Concept  Refinement,  Technology  Development,  System 
Development  and  Demonstration,  Production  and  Deployment,  and  Operations  and  Support 
phases.  At  each  phase,  the  Project  Manager  strives  to  attain  the  best-value  solution  for  a 
specific  requirement  for  his/her  platform.  If  multiple  Platform  Managers  have  similar 
requirements,  these  Platform  Managers  will  focus  on  obtaining  best  value  for  their  platforms 
within  the  constraints  of  their  resources  and  where  they  are  in  their  respective  lifecycle  phases. 

By  managing  the  seven  layers  of  the  Capabilities  Engine,  the  Capability  Manager  looks 
vertically  at  specific  knowledge  points  in  time  to  ensure  that  all  potential  solution  sets  are 
considered  across  multiple  platforms.  Formal  synchronization  points  at  which  the  Capability 
Manager  reports  success  against  a  specified  set  of  metrics  (cost,  schedule,  and  performance 
perspective)  can  be  achieved  across  multiple  programs.  The  Knowledge  Points  suggested  in 
this  paper  are  consistent  with  those  recommended  by  the  GAO  study  that  reviewed  the  NASA 
process  with  which  it  made  its  investment  decisions  (GAO.  (2005,  December).  Implementing  a 
knowledge-based  acquisition  framework  could  lead  to  better  investment  decisions  and  project 
outcomes.  Report  to  Congress.  Washington,  DC:  author). 

•  Knowledge  point  1  (KP1):  Resources  and  needs  match.  Knowledge  point  1  occurs 
when  a  sound  business  case  is  made  for  the  product.  According  to  the  GAO,  this 
requires  a  match  between  the  customer’s  requirements  and  the  product  developer’s 
available  resources  in  terms  of  knowledge,  time,  workforce,  and  money.  By 
synchronizing  all  seven  layers  of  the  Capabilities  Engine  at  this  point,  the  Capabilities 
manager  not  only  matches  requirements  with  resources  and  time,  but  ensures  the 
requirement  is  consistent  with  standards,  an  optimized  material  solution,  is  appropriate 
for  the  specific  organization  receiving  the  system,  and,  most  importantly,  is  consistent 
with  the  battle  space  within  which  the  material  solution  will  be  integrated. 

•  Knowledge  Point  2  (KP2):  Product  design  is  stable.  Design  stability  is  critical  to 
reducing  risk.  At  this  point,  the  Capability  Manager  continues  to  validate  that  all  the 
layers  of  the  Capability  Engine  are  synchronized  and  that  no  perturbations  from  any  of 
the  layers  has  caused  a  break  in  the  connection  from  the  Standards  layer  to  the 
Battlespace  layer.  Additionally,  the  Capabilities  Manager  ensures  that  the  solution(s)  for 
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the  initiating  requirement  are  still  consistent  with  the  warfighter’s  intent  and  that  the 
Platform  Managers  have  not  introduced  potential  dissimilarities  within  their  platform- 
strategic  plans  than  might  manifest  in  subsequent  phases. 

•  Knowledge  Point  3  (KP3):  Production  Processes  are  mature.  This  is  effectively  the 
Platform  Manager’s  milestone  C  point  at  which  the  risk  must  be  low  enough  to  precede 
into  production  contracting.  The  Capabilities  Manager  has  less  of  a  role  at  this  point — 
with  the  exception  of  continuing  to  monitor  the  capability  effects  across  the  layers  of  the 
Capability  Engine.  Configuration  management  of  the  platform  solution  is  critical  to 
maintaining  a  consistent  capability  across  the  force.  At  this  point,  if  the  Platform 
Manager  introduces  changes  to  the  production  architecture,  the  potential  exists  for  the 
synchronized  solution  to  “stovepipe”  into  a  unique  platform  solution.  The  Capability 
Manager  must  monitor  the  configuration  control  of  all  systems  across  multiple  Program 
Offices,  with  regard  to  each  other,  while  the  Program  Manager  must  continue  to  sustain 
and  improve  the  capability  on  the  unique  platform. 

Conclusion 

Viewing  material  acquisition  in  a  more  holistic  manner  and  striving  to  optimize  solutions 
for  specified  requirements  should  be  the  goal  for  all  the  Services.  Although  the  defense  budget 
is  low  with  regard  to  the  GDP  compared  to  past  points  in  history,  current-year  dollar  value  is  the 
highest  it’s  ever  been.  The  acquisition  process,  however,  continues  to  develop  fewer  systems 
at  greater  expense.  As  the  warfighter  views  the  battlespace  as  an  interconnected  environment 
of  mutually  supporting  systems,  our  Program  Management  process  must  also  adapt  to  this 
environment.  We  must  begin  to  view  systems  as  contributors  of  capability  in  which  systems  are 
mutually  supporting,  in  which  the  combined  effect  is  greater  that  the  individual  parts.  A 
Capability  Management  Process  that  supports  the  “traditional”  Program  Management  process 
needs  to  be  developed  to  inculcate  a  standardized  approach  to  managing  capability. 

The  thoughts  presented  in  this  paper  look  at  the  need  to  change  the  current  paradigm 
and  how  one  might  approach  a  capability  management  model.  The  problem  is  complex,  but  the 
need  for  a  solution  is  imperative.  Developing  an  execution  strategy  to  capability  management  is 
even  more  complex.  Understanding  the  need  to  optimize  and  synchronize  material  solutions  is 
merely  the  beginning.  Developing  an  organizational  construct  to  execute  this  mission  is  worthy 
of  continued  study  and,  ultimately,  of  implementation.  It  is  important  in  this  study  to  heed  the 
words  of  Sun  Tzu:  “avail  yourself  also  of  any  helpful  circumstances  over  and  beyond  the 
ordinary  rules”;  for  as  in  War,  the  systems  we  provide  the  warfighter  will  shape  the  outcome  of 
the  battle. 
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Abstract 

This  paper  documents  the  results  of  a  multi-year  Army  Materiel  Command-sponsored 
research  project  which  employed  a  structured  case  study  approach  to  examine  the  history  and 
processes  that  had  resulted  in  the  introduction  of  a  number  of  technology-based  Army  systems 
in  time  to  make  a  positive  contribution  to  the  outcome  of  Desert  Storm.  In  addition  to  the  fifteen 
case  studies  documenting  these  programs,  a  common  set  of  data  was  obtained  for  each  system 
studied.  These  data  were  analyzed  to  identify  factors  contributing  to  successful  systems 
development;  this  paper  contains  the  results  of  this  analysis. 
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Several  of  the  statistically  significant  relationships  found  involve  factors  that  are  related 
to  the  stability  of  the  program.  When  key  members  of  the  project  team  left  the  program  too 
early,  project  outcome  suffered.  Further,  both  project  funding  cutbacks  and  project  team  turn¬ 
over  negatively  correlated  with  the  quality  of  the  testing  program  and  the  timeliness  of  key  test 
events.  These  two  attributes  of  the  testing  program  also  had  the  strongest  correlation  with 
project  outcomes.  In  addition,  changes  in  systems  requirements  during  development  correlated 
with  poor  project  cost  performance.  Finally,  turn-over  in  key  user-representative  personnel 
correlated  negatively  with  system  performance  in  the  field.  A  central  conclusion  from  this  study 
is  that  shorter  development  cycle-times  favorably  correlate  with  key  project  outcome  variables, 
largely  by  minimizing  the  exposure  of  the  project  to  destabilizing  influences  which  were  also 
shown  to  correlate  negatively  with  these  same  outcome  variables. 

Keywords:  technology-based  Army  systems,  project  outcome,  system  requirements, 
development  cycle-time 

Introduction 

This  paper  documents  the  results  of  a  research  project  of  several  years’  duration  which 
employed  a  structured  case  study  approach  to  examine  the  history  and  processes  that  had 
resulted  in  the  introduction  of  a  number  of  technology-based  Army  systems  in  time  to  make  a 
positive  contribution  to  the  outcome  of  Desert  Storm.  The  15  case  studies  that  resulted  were 
developed  on  systems  ranging  from  the  M829A1  “silver  bullet”  to  the  GUARDRAIL  Common 
Sensor  and  the  APACHE  attack  helicopter. 


Research  Project  Information 


•  Principal  Sponsor:  Army  Material  Command 

•  Principal  Investigators:  Bill  Lucas  (MIT)  and  Dick 
Rhoades  (UAH) 

•  Research  Period:  September  1999  to  May  2004  (data 
analysis  and  report  preparation  continued  into  2005) 

•  Funding:  -$200,000 

•  Research  Purpose:  Examine  the  history  and  processes 
used  in  the  development  of  a  number  of  Army  systems 
which  made  a  positive  contribution  on  the  battlefield 
during  Desert  Storm 

—determine  factors  which  influence  success 
—prepare  case  studies 


Figure  1.  Project  Overview 
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Systems  Studied 


Svstem 

Researcher 

Commoditv  category 

APACHE  attack  helicopter 

Ference 

Aviation 

TADS/PNVS  (target  acquisition  and 
designation/pilot’s  night  vision  systems) 

Oelrich 

Aviation 

MLRS  rocket  system 

Sherman 

Missiles 

ATACMS  missile  system 

Romanczuk 

Missiles 

M40  chemical  protective  mask 

Ruocco 

Soldier  support 

Dismounted  microclimate  cooler 

Note:  Did  not  enter  production 

Ruocco 

Soldier  support 

Mounted  microclimate  cooler 

Ruocco 

Soldier  support 

M829-A1  armor-piercing  kinetic  energy 
tank  ammunition 

Mitchell 

Ammunition 

FOG-M  (fiber  optic  guided  missile) 

Note:  Did  not  enter  production 

Sherman 

Missiles 

T0W-2A  (Tube-launched  missile) 

Vessels 

Missiles 

AN/TAS  4  infrared  night  sight 

Granone 

Target  acquisition 

Joint  Stars  Ground  Station 

Sherman 

Intelligence 

Guardrail  common  sensor 

Sherman 

Intelligence 

PAC-2  (PATRIOT  anti-missile  system) 

Sherman 

Missiles 

HELLFIRE  missile  system 

Johansen 

Missiles 

Figure  2.  Systems  Studied 

The  case  studies  were  developed  through  the  use  of  structured  interviews  with  key 
participants  from  the  government/contractor  team  that  developed  each  system.  In  addition  to  the 
case  studies,  this  process  resulted  in  collection  of  a  common  set  of  data  for  the  systems  studied 
which  could  then  be  analyzed  to  identify  factors  contributing  to  successful  system  development. 
The  results  of  this  analysis  are  contained  in  this  paper.  Two  of  the  15  case  studies  examined 
systems  which  might  have  been  useful  on  the  battlefield  (based  on  the  views  of  Army  technical 
leaders),  but  that  failed  to  successfully  complete  development.  The  intent  of  including  failures  in 
the  research  was  to  provide  a  basis  for  distinguishing  factors  which  contributed  to  both 
successful  and  unsuccessful  system  developments.  While  they  are  useful  for  the  qualitative 
lessons  they  offer,  two  cases  are  inadequate  for  quantitative  analysis;  most  analysis  focuses  on 
the  13  successful  cases.  The  study  is,  therefore,  primarily  an  assessment  of  contributors  to  the 
relative  degree  of  success.5 


5  The  LeanTEC  project  was  a  four-year  study  of  the  development  and  transition  of  technology-dependent 
systems  in  the  aerospace  industry,  supported  by  a  cooperative  research  agreement  between  the  US  Air 
Force  Manufacturing  Technology  Office  and  The  Boeing  Company. 


rtusu'crufuicn.vrui. 
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Research  Methodology 


•  Army  RDEC  and  PM  leadership  nominated  systems 
which  either  did  or  could  have  impacted  Desert  Storm 

•  Researchers  (intended  to  be  “free”  Army  student  labor) 
selected  a  system  from  list  of  candidates 

•  “Structured  thesis”  approach  used  to  gather  comparable 
data  on  each  system  studied,  but  allow  researcher  to 
document  areas  of  particular  interest  in  each  case  study 

•  Modified  version  of  questionnaire  used  on  LeanTEC* 
was  administered  to  Army  and  contractor  development 
team  members;  researcher  integrated  responses 
--produced  composite  “best  answer”  questionnaire 
—produced  case  study  on  system  development 


15  systems,  13  produced  dictated  a  focus  on  relative  success  factors 


Figure  3.  Methodology  Employed 

The  heart  of  any  systematic  study  is  the  definition  of  a  common  outcome  measure  that 
allows  comparison.  The  obvious  path  was  to  compare  the  projects  and  systems  based  on  their 
performance  relative  to  their  agreed-upon  goals  and  requirements.  Each  project  had  a  budget, 
a  systems  procurement  cost  goal,  a  set  of  technical  requirements,  and  completion  dates.  In 
addition,  three  questions  of  performance  are  immediately  observable  and  easily  remembered  by 
project  managers:  Did  the  system  go  into  production?  Once  production  was  started,  were 
problems  found  that  required  that  further  engineering  changes  be  made?  And  did  the  system 
perform  well  in  its  use  in  Desert  Storm?  Structured  questions  were  used  to  ask  the  key 
government  and  industry  interviewees  about  how  well  their  projects  performed  in  these  areas, 
with  a  range  of  answers  that  characterized  how  badly  the  projects  had  missed  meeting  their 
objectives  if  they  had  not  been  completely  successful.  Each  of  these  outcomes  is  shown 
graphically  in  the  histograms  which  follow. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -476- 


Outcomes-Development  Budget 


underran  exceeded  exceeded 


Budget  Performance 


Figure  4.  Development  Budget 


Outcomes-System  unit  cost 


7 

6 

5 

4 

3 

2 

1 

O 


MLRS,  ATACMS, 

HB_LFIRE,TADS/PNVS,  M829A1  and 


Mounted  rticrocooler 


Guardrail,  PAC-2,  TOW  2A, 

Night  Sight,  APACHE  and 
M4Q  mask 


Met  or 
exceeded 


Came  close 
to  goal 


Joint  Stars 


Far  short 
of  goal 


Cost  Performance 


Figure  5.  System  Unit  Cost 
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Outcomes-Technical  performance 


3 

Z 


10 

9 


8 

7 


6 


5 


4 

3 

2 

1 

0 


Met  or  Came  close 

exceeded 


Achievement  of  Technical  Goals 


Figure  6.  Technical  Performance 


Outcomes-Delay  in  transitioning  to 
production 


8 

7 

6 

5 

4 

3 

2 

1 

O 


Vehicle  microcooler, 

GUARDRAIL,  ATACMS, 

PAC-2,  MLRS,  M829  A1, 

Niaht  siaht 

APACHE,  HELL  FIRE 

Joint  Stars, 

M40  Mask 

No  delay  1-6  months  7-12  months  year  +  late 
late  late 

Development  to  production  delay 


Figure  7.  Delay  in  Transitioning  to  Production 
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Outcomes-Changes  in  production 


9 

8 

7 

6 

5 

4 

3 

2 

1 

0 


Joint  Stars.  TADS/PNVS. 

TOW-2  A,  PAC-2,  MLRS, 

M829  A1 ,  Night  Sight, 

HFI  1  FIRF 

GUARDRAIL, 

A  1 A  CMS , 

Mounted  micro- 

cooler 

APACHE,  M40  Mask 

No  or  almost 
no 


Minor 


Significant 


Engineering  changes  in  production 


Figure  8.  Changes  in  Production 


Six  of  the  outcome  measures  mentioned  above  were  used  to  create  a  scale  that  scores 
the  (system)  projects  from  zero  to  six  according  to  the  number  of  key  outcomes  a  project 
achieved.  If  a  project  was  (1)  transitioned  to  production  on  time,  (2)  developed  within  budget, 
(3)  had  no  late  engineering  changes,  met  both  (4)  the  goals  for  system  unit  costs  and  (5)  its 
technical  requirements,  and  encountered  (6)  no  difficulties  when  it  was  deployed  in  the  field,  it 
was  awarded  (the  maximum)  six  points. 


Outcomes-lntegrated  Scale 


Outcome  scale 


Scale:  Sum  of  number  of  preferred  outcomes  using  six  outcome  metrics 


Figure  9.  Integrated  Scale 
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Table  1  (next  page)  contains  summary  information  on  the  15  systems  studied.  For  each 
system,  this  table  also  contains  information  on  the  duration  of  the  development  phase  of  the 
program  and  a  summary  of  the  project  manager’s  description  of  the  most  difficult  problem 
encountered.  It  is  interesting  to  note  that  lack  of  sustained  user  support  for  the  requirement  the 
system  was  intended  to  satisfy  was  mentioned  as  the  most  difficult  problem  for  the  two  failures, 
but  user-related  issues  were  not  identified  for  any  of  the  successful  development  cases. 


Svstem/case 

Development 

duration  (months) 

PM’s  most  difficult 

problem 

Key 

outcomes 
achieved  (0- 

6 1 

APACHE  attack 
helicopter 

108 

Control  of  production 
costs;  influenced  by 
integration  plant  location 
choices 

1 

TADS/PNVS  (target 
acquisition  and 
designation/pilot’s  night 
vision  systems) 

~36 

Cost  growth  in 
development 

3 

MLRS  rocket  system 

33 

Establishing  and 
managing  four-nation 
cooperative  development 
program 

6 

ATACMS  missile  system 

37 

Key  vendor  went  out  of 
business 

6 

M40  chemical  protective 
mask 

~48 

Immaturity  of  critical 
technologies 

2 

Dismounted  microclimate 
cooler 

Note:  Did  not  enter  full 
development 

Not  applicable 

Lack  of  stable  user 
requirements  due  to 
immaturity  of  technology 

Not 

applicable 

Mounted  microclimate 
cooler 

~24 

Key  vendor  failed  to 
support  integration 
schedule 

5 

M829-A1  armor-piercing 
kinetic-energy  tank 
ammunition 

~36 

Achieving  needed 
innovation  in  system 
design 

6 

FOG-M  (fiber-optic 
guided  missile) 

Note:  Did  not  complete 
development 

Not  applicable; 

Lack  of  sustained  user 
support 

Not 

applicable 

TOW-2A  (Tube-launched 
missile) 

48 

Stability  of  threat  armor 
requirements 

3 

AN/TAS  4  infrared  night 
sight 

~24 

Selection  of  unqualified 
vendor  and  split 
management 
responsibility 

4 
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Joint  Stars  Ground 

Station 

105 

Cost  and  schedule 
growth/delivering 
complex  software 

1 

Guardrail  common  sensor 

~24 

Complexity  of  integration 
of  mission  equipment 

3 

PAC-2  (PATRIOT  anti¬ 
missile  system) 

~52 

Early  fielding  to  meet 
SCUD  missile  threat 

2 

HELLFIRE  missile 
system 

~84 

Adversarial  relationship 
between  key  vendor  and 
prime 

3 

Table  1.  Summary  Case  Information 


Standard  statistical  analysis  procedures  appropriate  for  this  number  of  cases  and  type  of 
data  were  used  to  identify  and  evaluate  correlations  between  the  factors  studied  and  the  several 
outcome  variables,  and,  in  some  cases,  among  the  factors.  The  results  of  these  analyses  are 
summarized  in  Table  2.  The  testing/simulation  and  technological  maturity  factors  were  included 
because  of  their  identification  in  recent  Government  Accounting  Office  studies  as  key 
determinants  of  success. 


Factor 

Relationships  Found/Comments 

1.  Project  team  characteristics 
and  practices: 

— leadership 

Team  leader’s  perceived  ability  to  obtain  resources,  his/her 
breadth  of  experience  and  ability  to  resolve  technical  issues  all 
are  positively  related  to  reduced  engineering  changes  during 
production  and  to  completing  development  within  budget. 

— staffing 

Low  turnover  in  key  project  team  members  relates  positively  to 
completing  development  within  budget,  to  meeting  system  unit 
cost  targets  and  to  achieving  system  performance  objectives. 

2.  Role  of  government  S&T 
organizations 

Army  labs/centers  were  typically  actively  involved  in  both  pre¬ 
development  and  development  phases,  actively  involved  in 
both  successes  and  failures,  and  actively  involved  in  both  short 
and  long  developments. 

3.  Testing  and  simulation 
approach 

Validating  component  and  system  maturity  at  the  right  time  in 
the  program  relates  positively  to  completing  development  within 
budget,  to  meeting  system  unit  cost  targets  and  to  successful 
performance  in  the  field.  The  quality  of  the  testing  and 
simulation  conducted  relates  positively  to  reduced  engineering 
changes  during  production  and  to  meeting  system  unit  cost 
targets. 

4.  Importance  of  stability: 

— funding 

Funding  uncertainty  was  related  to  increased  turnover  in  key 
project  team  members  and  the  need  to  deal  with  changes  in 
testing  plans  and  other  project  structure  issues. 

— system  requirements 

Changes  in  system  requirements,  particularly  during  the  middle 
of  development,  relate  to  an  increase  in  late  engineering 
changes  and  negatively  to  project  success  in  meeting  its  goals 
for  systems  costs. 

— key  user  (TRADOC) 

Changes  in  key  TRADOC  personnel  during  development 

n.-.i'.uMiAftr.  v  hsiu,, 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -481  - 


personnel 

relates  to  less  successful  performance  in  the  field. 

5.  Timely  communication  of 
problems 

Nearly  all  cases  described  timely  communication  of  problems 
from  contractor  to  government  PM  and  from  government  PM  to 
Army  leadership. 

6.  Importance  of  technology 
maturity  (TRLs) 

Maturity  of  critical  technologies  used  in  systems  studied,  as 
measured  by  TRLs,  was  similar  to  that  found  in  previous 

LeanTec  study  of  small  electronics  projects.  No  positive 
correlation  found  between  higher  TRLs  at  the  start  of 
development  and  most  outcome  variables. 

Table  2.  Summary  of  significant  relationships 


Several  of  the  statistically  significant  relationships  involve  factors  that  are  related  to  the 
stability  of  the  program.  When  key  members  of  the  project  team  left  the  program  too  early, 
project  outcome  suffered.  Further,  both  project  funding  cutbacks  and  project  team  turn-over 
negatively  correlated  with  the  quality  of  the  testing  program  and  the  timeliness  of  key  test 
events.  These  two  attributes  of  the  testing  program  also  had  the  strongest  correlation  with 
project  outcomes.  In  addition,  changes  in  systems  requirements  during  development  correlated 
with  poor  project  cost  performance.  Finally,  turn-over  in  key  user-representative  personnel 
correlated  negatively  with  system  performance  in  the  field. 


Destabilizing  Influences 


Variable 

Timing  Implications 

1.  Reductions  in  project 
funding 

Potential  for  change  in  administration  every  48  months;  typical 
turn-over  in  key  military  leaders  occurs  every  24-36  months. 
Potential  change  in  key  Congress  positions  every  24  months; 
likelihood  increases  with  development  duration 

2.  Uncertainty  in  project 
funding 

Potential  for  change  in  administration  every  48  months;  typical 
turn-over  in  key  military  leaders  occurs  every  24-36  months.  . 
Potential  change  in  key  Congress  positions  every  24  months; 
likelihood  increases  with  development  duration. 

3.  Change  in  system 
requirements 

Changes  in  the  threat  environment  occur  unpredictably,  but 
become  more  likely  with  longer  development  durations. 

Changes  in  doctrine  and  system  requirements  follow  a  similar 
pattern. 

4.  Change  in  key  user 
representatives 

Typical  turn-over  in  such  key  military  positions  occurs  every 
~36  months 

5.  Change  in  key  project  team 
members 

Typical  turn-over  in  military  acquisition  positions  occurs  every 
~36  months.  Longer  development  durations  present  more 
opportunities  for  career  moves  on  the  part  of  key  civilian  team 
members 

Figure  10.  Destabilizing  Influences 

Taken  together,  these  several  relationships  strongly  suggest  that  stability  of  program 
resources  and  objectives  is  a  very  powerful  influence  on  the  relative  success  of  the  project.  In 
reflecting  on  this  array  of  instabilities  that  could  impact  a  system  development,  it  became  clear 
that  they  had  at  least  one  thing  in  common:  The  longer  a  system  stayed  in  development,  the 
greater  chance  it  had  to  experience  one  or  more  of  these  program  destabilizing  events.  Or, 
stated  another  way,  shorter  system  development  cycles  should  result  in  better  project 
outcomes.  When  this  hypothesis  was  tested  by  examining  the  correlation  between  the  system 
development  durations  and  the  aggregate  outcome  scale  (See  the  data  in  Table  1),  a  strong 
correlation  was  found.  A  central  conclusion  from  this  study  is,  therefore,  that  shorter 
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development  cycle-times  favorably  correlate  with  key  project  outcome  variables — largely  by 
minimizing  the  exposure  of  the  project  to  destabilizing  influences  which  have  also  been  shown 
to  correlate  negatively  with  these  same  outcome  variables. 


Central  Conclusion 


Shorter  development  cycle  times  favorably 
correlate  with  key  project  outcome  variables, 
largely  by  minimizing  the  exposure  of  the  project 
to  destabilizing  influences 


Length  of  Project  Development  and  Project  Performance 

(Average  number  of  successful  outcomes) 

Three  years 

Over  3  years  or  less  Sig.  at 
Length  of  development  2.00  4.71  .002 


Figure  11.  Conclusion 

Whether  or  not  a  change  to  selecting  projects  with  shorter  development  times  is  made, 
the  Army  could  do  more  to  stabilize  the  guidance  and  resources  given  to  both  shorter  and 
longer  development  projects.  Acting  alone,  the  Army  could  do  more  to  map  rotating  personnel 
assignments  and  other  sources  of  TRADOC  change  to  project  development  cycles.  Since  it 
appears,  as  is  widely  believed,  that  changes  in  systems  requirements  made  once  projects  move 
beyond  early  development  will  almost  certainly  hurt  project  performance,  the  Army  could 
eliminate  all  but  the  most  critically  important  of  such  changes.  Both  through  contract  language 
and  informal  management  practices,  the  Army  could  work  with  its  contractors  to  provide  better 
continuity  of  development  project  staffing. 

The  defense  acquisition  community  has  long  recognized  that  lengthy  systems 
development  times  are  disadvantageous.  Sometimes  the  associated  negatives  have  been 
phrased  in  program  instability  terms;  this  study  certainly  provides  strong  empirical  support  for 
those  who  hold  these  beliefs.  Over  the  years,  a  number  of  initiatives  have  been  attempted  to 
shorten  development  cycles,  with  limited  success  where  complex  systems  were  involved.  The 
current  approach  is  referred  to  as  “spiral  development”;  its  basic  concept  is  to  get  a  useful,  if 
limited,  capability  in  the  field  quickly  and  then  introduce  additional  technology-based  capabilities 
through  further  “spirals”  of  development.  This  approach  appears  to  be  in  keeping  with  the 
implications  of  this  study’s  central  conclusion. 
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Abstract 

The  DoD’s  evolutionary  acquisition  policy  is  directed  against  project  risk,  but  bears 
inherent  risks  of  its  own.  The  DoD  policy  for  evolutionary  acquisition  mandates  multiple  product 
releases  via  spiral  (i.e.,  amorphous  &  unplanned)  or  incremental  (i.e.,  defined  &  deferred) 
development  methodologies  for  all  programs.  All  amorphous  spirals  eventually  become 
definitive  increments.  Incremental  development  entails  the  deliberate  deferral  of  work  to  a 
subsequent  phase.  Computational  organizational  modeling  using  systems  dynamics  reveals 
that  this  methodology  introduces  more  concurrency  during  development,  and  more  variety  in 
production.  The  result  is  earlier  delivery  of  the  first  increment,  but  with  later  and  more  costly 
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delivery  of  subsequent  increments  than  if  conducted  via  a  single-step  methodology. 

Curtailments  of  scope  by  the  exclusive  use  of  mature  technology  enable  more  effective  delivery 
of  the  first  increment,  further  illustrated  by  two  case  studies.  Duplication,  rework,  transaction 
costs,  decision  backlog  and  error  are  causes  of  inefficiency  in  the  successive  increments. 
Production  variety  and  mixed  configurations  produce  obvious  implications  for  logistical 
supportability,  training,  failure  causality,  compatibility  and  interoperability,  etc.  Further,  certain 
attributes  of  hardware  products  might  help  determine  the  suitability  of  this  development 
methodology.  Products  that  are  nearly  immutable,  which  have  binary  requirements  for  key 
capabilities,  require  man-rating,  or  are  maintenance-intensive  may  not  be  good  candidates  for 
incremental  development.  Mutable  products  with  costless  production,  continuous  requirements, 
low  maintenance,  or  time  criticality  are  more  likely  to  reap  advantages  from  this  development 
approach.  While  modular  open  systems  architecture  facilitates  system  adaptation,  modularity 
itself  does  not  necessarily  create  evolutionary  advantages  due  to  relative  modular 
interdependency.  Program  managers  must  be  aware  of  the  inherent  risks  of  these  agile 
acquisition  methods  and  take  additional  steps  to  balance  them  with  appropriate  planning  and 
resources,  disciplined  change-control  measures,  organizational  accommodations  and 
accountability  for  configuration  management. 

Keywords:  Evolutionary  acquisition,  spiral  development,  incremental  product 
development,  Javelin,  ATACMS,  agile  development  methodologies,  computational 
organizational  modeling,  modularity. 

Introduction — The  Inevitability  of  Change 

We  are  told  in  Diogenes  Laertius's  Lives  and  Opinions  of  Eminent  Philosophers  (early 
3rd  century)  that  the  Greek  philosopher  Heraclitus  (c.535  -  475  BC)  was  the  first  to  observe  and 
say,  “Everything  flows;  nothing  stands  still,” — the  popular  derivation  of  which  is,  “The  only 
constant  is  change.”  Indeed,  everything  does  seem  to  change,  evolve  and  give  rise  to  variety  in 
the  world.  Since  his  work  in  the  1830s,  Charles  Darwin  receives  much  of  the  credit  for 
furthering  a  theory  of  biological  evolution.  While  not  the  first  to  have  the  idea,  he  associated 
observations  of  species  variety  on  the  island  of  Galapagos  with  species  environment ,  and 
suggested  that  nature  selected  the  variations  that  were  the  fittest  (Darwin,  1859).  In  its  time 
(and  even  since),  the  idea  was  considered  radical  and  a  threat  to  the  religious  and  social  order 
of  things.  Mere  variety  itself  can  be  controversial,  since,  paradoxically,  variety  is  appreciated  in 
some  domains  (see  the  writings  of  William  Cowper,  1731-1800)6  and  abhorred  in  others  (Neave, 
2000,  March  2). 7  At  the  core  of  the  subject  of  evolutionary  acquisition  are  ideas  and  phenomena 
about  variety  and  change.  As  a  policy  for  system  development,  it  is  controversial  too.  As  with 
Darwinian  concepts,  product  evolution  involves  information  transfer,  interaction  with  the 
environment  and  unpredictability  of  change  outcomes.  But  unlike  evolutionary  biology,  product 
variations  and  selections  occur  frequently  and  are  non-random.  Program  managers  typically 
seek  stability — in  program  requirements,  in  funding,  in  system  design,  and  in  production 
configuration.  But  it  seems  the  only  constant  is  change.  Everything  changes  and  evolves  over 


6  See  also:  Kerr  (1 979,  p.  65)  about  the  basic  human  need  for  variety  and  complexity.  Ashby’s  Law  of 
Requisite  Variety  states  that  the  internal  regulatory  mechanisms  of  a  system  must  be  as  diverse  as  its 
environment  in  order  to  cope  with  the  variety  of  challenges  imposed  by  it  (Ashby,  1 960). 

2  “Variation  is  nasty:  it  makes  things  difficult,  unpredictable,  untrustworthy:  bad  quality.”  “In  a  big  way,  bad 
quality  means  too  much  variation,  good  quality  means  little  variation.” 
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time.  Much  of  what  the  authors  have  found  in  their  following  research  on  spiral  development  and 
project  management  is  about  how  managers  must  cope  with  product  variety  and  change.  Using 
case  study  analyses,  review  of  current  subject  literature,  and  computational  modeling,  the  focus 
of  our  research  was  to  ascertain  the  acquisition  management  implications  of  spiral 
development,  obtain  lessons  learned  in  past  programs  as  applicable  to  future  development 
efforts,  model  and  simulate  projects  using  different  acquisition  approaches,  derive  predictions 
and  make  recommendations  to  project  managers  for  the  effective  and  efficient  harnessing  and 
implementation  of  spiral  development. 

Background 

Projects  have  long  been  defined  as  unique  and  temporary  enterprises,  as  opposed  to 
common  and  ongoing  operations.  The  latest  (2004)  version  of  the  Project  Management  Body  of 
Knowledge  ( PMBOK)  increased  its  emphasis  upon  the  term  “progressive  elaboration”  to 
describe  a  third  fundamental  characteristic  of  all  projects.  It  means,  “developing  in  steps  and 
continuing  by  increments;  worked  out  with  care  and  detail;  developed  thoroughly”  (PMBOK, 
2000;  PMBOK,  2004,  p.  6).  This  term  relates  to  project  uncertainty  and  describes  the  eventual 
realization  of  project  scope  only  after  multiple  iterations  of  planning.  The  PMBOK  asserts  that 
progressive  elaboration  is  both  a  necessary  characteristic  of  projects  (occurring  throughout  their 
lifecycles),  as  well  as  a  technique  for  development  of  product  specifications.  It  is  accomplished 
via  the  learning  that  takes  place  over  time  as  project  ambiguity  resolves,  so  that  project  scope 
becomes  more  explicit  and  detailed  (as  opposed  to  “requirements  creep,”  which  is  considered 
uncontrolled  change).  The  PMBOK  later  asserts  that  change  in  the  course  of  projects  and 
products  is  inevitable,  and  mandates  the  need  for  a  disciplined  change-control  process  to 
control  its  impacts — from  inception  to  completion  (PMBOK,  2004,  p.  119). 

There  are  many  new  DoD  terms  for  project  management  and  product  development 
methods.  DoD  promulgated  evolutionary  acquisition  (EA)  as  policy  in  2000,  and  soon  after, 
spiral  development  for  the  preferred  acquisition  strategy  of  all  materiel.  EA’s  goal  is  to  phase 
requirements  and  provide  capability  sooner.  But  there  has  been  confusion  over  terms,  despite 
further  elaboration  and  even  codification  in  statute,  and  it  still  persists  today,  along  with  a  lack  of 
full  understanding  of  many  policy  implications — especially  some  inherent  risks.  EA  operationally 
means  there  will  always  be  multiple  product  releases  of  an  item. 

The  policy  thrust  is  primarily  about  the  reduction  of  product  cycle-time  within  an 
uncertain  environment,  by  exclusively  using  mature  technology.  The  DoD’s  requirements 
process  has  also  followed  with  “evolutionary”  requirements  documents — a  new  idea. 

Uncertainty  is  the  usual  realm  of  program  managers,  especially  in  defense  systems,  and  is 
usually  dealt  with  by  seeking  best  information.  Earlier  reform  initiatives  were  aimed  at 
overcoming  information  gaps  and  technology  lag.  The  1990’s  Integrated  Product  and  Process 
Development  (IPPD)  initiative  was  about  gaining  collective  wisdom  for  early  and  complete 
requirements  realization.  As  concerns  over  DoD  acquisition  program  costs  and  cycle-times 
continue  in  the  current  mid-2000s  era,  the  DoD  has  not  abandoned  the  use  of  IPPD.  But  by 
embracing  evolutionary  requirements  and  acquisition,  it  has  acknowledged  that  information  will 
never  be  complete,  either  from  stakeholders  or  with  regard  to  ever-changing  technology.  It  now 
implicitly  concedes  that  developers  will  learn  about  their  design  over  time  (“requirements 
realization”),  and  users  will  accretively  gain  knowledge  about  how  they  can  better  use  the  new 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -486- 


capability  (“product  discovery”).8  Thus,  a  major  paradigm  shift  for  product  development  has 
occurred  in  the  DoD:  from  a  collaborative  quest  to  capture  and  address  all  requirements  early 
on,  to  an  allowance  of  eventual  requirements  discovery  with  full  attainment  only  after 
visualization,  feedback  and  environmental  changes  occur  along  the  way. 

The  Enabler:  Mature  Technology  Reduces  Risk 

This  is  not  to  say,  however,  that  the  DoD  has  in  its  policy  embraced  technological 
uncertainty  for  the  commencement  of  advanced  development.  Quite  the  contrary — for  at  the 
very  heart  of  the  evolutionary  acquisition  strategy  is  the  requirement  for  the  exclusive  use  of 
mature  technology  to  reduce  technology  risk.  The  impetus  for  this  undoubtedly  lies  in  the  body 
of  work  by  the  Government  Accountability  Office  (GAO)  over  the  last  ten  years,9  which  has 
obviously  and  greatly  influenced  the  DoD  5000  series.  The  GAO  encourages  the  use  of 
knowledge-based  processes  and  specifically  separates  technology  development  from  product 
development.  It  argues  that  shorter  product  cycle-times  are  the  hallmark  of  program  success 
and,  therefore,  should  be  limited  to  five  years  for  more  frequent  introduction  of  new  technologies 
into  weapon  systems,  speeding  them  to  the  warfighter.  We  note  that  this  is  not  much  longer 
than  the  average  development  time  for  a  new  model  of  automobile — typically  3-4  years — which 
occurs  in  a  very  mature  and  cyclical  industry  (Kim,  2002,  June).  The  GAO’s  target  may  ignore 
the  significantly  greater  amount  of  technology  development  required  in  many  DoD  projects 
compared  with  most  automobile  development  projects. 

Most  emphasized  by  the  GAO  (in  the  many  reports  reviewed  by  these  authors)  is  the 
aspect  of  technology  maturity  before  commencement  of  advanced  development.  The  Office 
applies  a  1-through-9  rating  scale  of  technology  readiness  levels  (TRL)  that  was  developed  by 
the  National  Aeronautics  and  Space  Administration,  adopted  by  Army  and  Air  Force  research 
laboratories,  and  recently  implemented  in  the  DoD  5000  series  (in  particular,  the  Defense 
Acquisition  Guidebook — formerly  DoD  5000.2R ).  Until  recently,  the  DoD  had  no  specific 
requirements  for  use  of  TRLs,  but  levels  6  and  7  now  satisfy  its  guidelines  for  technology 
maturity  at  Milestone  B.  TRL  6  states  that  the  technology  has  been  demonstrated  in  a  relevant 
environment  (simulating  the  key  aspects  of  the  operational  environment),  and  TRL  7  is  its 
demonstration  in  an  operational  environment  (that  which  addresses  all  operational  requirements 
and  specifications  required  of  the  final  system,  to  include  platform/packaging).  The  GAO  clearly 
prefers  TRL  7  as  the  level  of  technology  maturity  that  will  represent  a  low  and  satisfactory  risk 
for  starting  product  development  (GAO,  2005,  November  15).  The  Office  acknowledges  that 
users  may  not  initially  receive  the  ultimate  capability  under  this  approach,  but  that  the  initial 
capability  will  arrive  predictably  sooner  and  cheaper  (GAO,  2005,  November  15). 

In  some  respects,  developing  only  mature  technology  as  a  fundamental  program 
requirement  is  similar  to  an  earlier  attempt  to  constrain  project  scope.  Cost  as  an  Independent 


8  The  authors’  terminology  for  what  has  so  often  been  observed  from  their  experiences.  Most  of  us  have 
long  known  that  full  realization  of  requirements  and  visualization  of  the  product  often  takes  multiple 
iterations  of  design,  with  feedback  loops  from  modeling  and  testing  activities.  And  sometimes  the 

customer  doesn’t  fully  realize  what  can  be  done  with  the  product  until  it  is  in  hand.  We  call  that  product 
discovery,  and  the  authors  can  cite  several  examples  of  this  in  both  commercial  and  defense  applications 
(i.e.,  cell  phones  as  improvised  explosive  device  triggers,  etc.). 

9  See  in  particular:  GAO/NSIAD-98-56;  GAO/T-NSIAD-98-123;  GAO/NSIAD-99-162;  GAO/T-NS I  AD-99- 
116;  GAO/T-NSIAD-00-1 37;  GAO-01-288;  GAO-02-701 ;  GAO-03-57;  GAO-04-53. 
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Variable  (CAIV)  was  an  acquisition  reform  initiative  that  emerged  in  1995  as  a  means  of  trading 
scope,  or  system  performance,  to  achieve  cost  objectives.  It  was  one  of  very  few  initiatives  that 
were  oriented  on  what,  not  how  (i.e.,  processes)  the  DoD  acquires  its  materiel.10  To  date,  its 
actual  savings  benefit  has  been  difficult  to  quantify,  and  qualitative  measures  have  shown  mixed 
results  (RAND,  2005).  Requirement  attainment  objectives  and  thresholds  were  another  way  to 
facilitate  performance  trades  for  cost. 

When  fully  realized,  it  is  the  exclusive  use  of  mature  technology  in  system  development 
programs  that  is  the  key  enabler  of  evolutionary  acquisition  strategy,  facilitating  the  rapid 
transformation  of  applied  technology  to  end-item  capability.  Thus,  it  is  the  third  of  three  principal 
observations,  all  of  which  are  paradigm  shifts,  that  we  have  recently  observed:  (1 )  that  the  DoD 
would  now  mandate  program  strategies  for  all  programs  to  have  multiple  product  releases  of  the 
same  item,  (2)  that  requirements  would  be  deferred  or  allowed  to  evolve  over  time,  and  (3)  that 
high  levels  of  technological  maturity  would  be  requisite  for  commencement  of  advanced 
development,  with  an  intended  reduction  of  technical  risk  (and  thus,  project  schedule) 
(USD(AT&L),  2003a,  May  12,  Enclosure— Additional  Policy  El. 14). 

Policy  and  Implementation  Concerns 

But  there  are  questions  and  concerns  about  these  major  shifts  that  several  authors  have 
raised.  Still  a  relatively  new  policy,  observations  and  realizations  about  the  outcomes  of 
evolutionary  acquisition  and  spiral  development  are  only  just  beginning  to  emerge,  and  will 
continue  to  surface  until  at  least  several  major  programs  go  through  their  entire  lifecycle  in  this 
way.  Sylvester  and  Ferrera  (2003)  provided  some  insight  into  the  challenges  and  obstacles  of 
evolutionary  acquisition  implementation — not  from  program-office  level — but  from  the 
perspective  of  strategic  policy-makers  and  subscribers  at  the  Office  of  the  Secretary  of  Defense 
(OSD)-level  during  their  struggle  to  adopt  the  policy.  In  short,  the  authors  explained  the 
aforementioned  confusion  and  ambiguity  of  the  policy  as  it  evolved  from  1983  toward  final 
promulgation  in  2000,  and  then  described  the  conflict  areas  caused  by  shifts  in  power  among 
the  organizational  fiefdoms  in  the  OSD  and  other  affected  institutions  (i.e.,  Congress  and  the 
defense  industry).  In  particular,  they  exposed  the  following  major  stakeholder  communities  and 
their  respective  areas  of  concern  about  evolutionary  acquisition: 

•  Congress — loss  of  control  over  DoD  programs  via  specific  and  informed  authorization 
and  approval;  the  inability  to  keep  the  DoD  accountable;  unknown  implications  of 
requirements  and  budget  flexibility  required  for  evolutionary  acquisition. 

•  Military  Departments — need  to  protect  own  acquisition  programs  and  share  of  the  DoD 
budget;  retention  of  funding  for  follow-on  capability  increments;  increased  oversight; 
downstream  logistics  of  multi-configuration  products. 

•  Defense  Industry — disruptions  to  commercial  processes  and  traditional  approaches  to 
business;  competition  for  follow-on  increments;  lower-rate  production  runs  after  shorter 
R&D  efforts. 


10  Some  may  also  assert  that  the  moratorium  against  MILSPECS  was  similar  in  its  thrust  to  reduce 
unnecessary  work  scope,  but  we  believe  specifications  to  be  as  much  prescriptive  (i.e.,  “how”)  as  they  are 
descriptive. 
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•  Comptroller — controlling  programs  and  holding  them  accountable;  unknown  implications 
of  requirements  and  budget  flexibility  required  for  EA  (program  and  budget  “gaming”  by 
services);  “full  funding”  policy11  versus  open-ended  requirements  and  fund  streams. 

•  Requirements/Users — sub-optimum  capability;  priority  of  what  is  needed  versus  what  is 
currently  attainable;  loss  of  follow-on  increments. 

•  Test  and  Evaluation — loss  of  discipline  and  assurance  of  operational  effectiveness  & 
suitability;  lack  of  comprehensive  testing  before  several  low-rate  production 
configurations  are  released. 

We  have  also  had  tactical  (implementation)  concerns  about  excessive  decision 
bureaucracy  (number  of  DAB  reviews — see  Figure  1),  organizational  challenges  from  multiple 
and  concurrent  development  efforts,  outdated  technology  at  release,  funds  forecasting, 
transaction  costs,  and  maintenance  of  subsequent  increment  priority.  It  is  these  phenomena 
that  we  have  modeled  with  computational  organizational  design  tools,  which  will  be  discussed 
later. 


1996  and  2003  DoD  5000  Models 
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Figure  1.  Comparison  of  1996  and  2003  Acquisition  Framework  Models 


11  The  authors  explain  the  dual  meanings  of  this  term  later  in  this  discussion. 
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The  Costs  and  Benefits  of  Variety — and  the  Need  for  Control 

Evolutionary  acquisition  methodologies,  in  addition  to  potentially  adding  more 
concurrency  during  development,  increase  variety  in  production.  Both  concurrency  and  variety 
are  elements  of  complexity  and  program  risk.  Variety  adds  complexity  in  production  and  is 
costly  for  hardware  owners  and  manufacturers  alike.  Traditional  views  about  late  design 
changes  are  negative,  except  for  producibility  enhancements  and  savings  or  correction  of 
design  flaws.  But  market  consumers  often  need  items  in  rapid  cycle-times  and  appreciate 
product  differentiation.  In  support  of  EA  policy,  the  GAO  has  used  product  examples  such  as 
commercial  vehicles.  For  the  most  part,  we  regard  these  commercial  products  as  relatively  “low- 
tech”  on  a  comparative  scale  of  DoD  system  complexity  and  capability.  Moreover,  we  feel  the 
GAO  may  ignore  some  very  important  aspects  of  ownership,  since  the  DoD  is  unique  as  an 
outsourcer  of  capitol  projects  for  internal  use,  and  has  unique  requirements  against  competitive 
threats  in  combat  environments. 

Control  measures  are  used  to  manage  risk.  One  way  of  coping  with  the  complexities  of 
variety  in  ownership  is  via  organizational  and  individual  accountability.  A  recent  example  of 
successful  control  of  rapid  change  lies  in  the  Acoustic  Rapid  COTS  Insertion/Advanced 
Processing  Build  (A-RCI/PB)  program.  In  this  vital  program  for  sustainment  of  submarine 
acoustic  sensing  superiority,  a  series  of  hardware  and  software  upgrades  were  planned  and 
executed  in  rapid  succession.  Each  emerged  with  advancement  in  capability,  keeping  pace  with 
technology  and  competitive  threats,  facilitated  by  rigorous  control  of  interfaces,  standards  and 
protocols  (Boudreau,  2006). 

Many  other  useful  theorems  on  systems  complexity,  change  and  control  exist  that  may 
be  helpful  for  practitioners  to  consider,  but  are  beyond  the  general  scope  of  our  research. 

Do  Product  Attributes  Affect  Spiral  Applicability  and  Outcomes? 

Spiral  development  as  a  universal,  “one-size-fits-all”  strategy  may  not  always  be 
appropriate.  Perhaps  the  foremost  reservation  is  the  appropriateness  of  the  spiral  development 
process  for  all  project  sizes  and  product  commodities  in  toto,  and  the  application  of  the  spiral 
process  to  hardware  products  versus  Boehm’s  original  and  most  relevant  application  of  this 
development  approach  toward  software.12  We  speculate  whether  certain  product  characteristics 
might  determine  spiral  development  method  applicability,  and,  thus,  may  offer  important 
considerations  for  project  planners. 

•  Mutability  simplifies  change,  and  spiral  development  was  conceived  for  the  most 
malleable  of  products:  “soft”  ware,  which  is  virtually  costless  in  production.  Multiple 
product  increments  do  not  often  appear  in  large,  static,  singular  projects  such  as 
bridges,  highways,  skyscrapers,  or  in  other  project  areas  that  have  typically  long  lead 
times  or  product  cycles,  such  as  feature-length  films,  pharmaceuticals,  etc.  These  are 
what  we  call  nearly  immutable  products  and  are  much  different  than  smaller  projects 
(like  small  application  software  development)  with  much  shorter  development  periods. 


12  And  the  authors  will  be  quick  to  acknowledge  that  software  is  indeed  a  huge  and  growing  part  of 
hardware  systems  large  and  small.  Still,  the  spiral  development  framework  in  current  literature  applies 
overwhelmingly  to  the  realm  of  software,  not  hardware. 
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•  Cycle-time  and  Phase  Concurrency.  Akin  to  relatively  mutable  or  immutable  products, 
we  have  observed  the  successive  product  upgrades  visible  in  long-running  aircraft 
programs  (UH-60  Blackhawk  and  C-130  Hercules  as  examples)  in  which  there  are 
periods  of  production  configuration  stability,  followed  by  improvement  efforts,  followed  by 
another  stable  use  period.  Cycle-time  for  the  development  of  each  increment,  and  the 
relatively  successive  or  concurrent  phasing  of  the  follow-on  increments,  will  have  a 
definite  impact  on  program  structure,  budgeting,  project  complexity,  and  organizational 
issues,  etc.  For  reasons  that  we  will  bring  forth  in  our  section  on  the  computational 
modeling  of  spiral  development,  we  have  concerns  about  the  conceptualization  of  spiral 
development  programs  with  continuous  and  highly  concurrent  phasing  of  development 
increments.  We  suggest  that,  though  concurrency  is  a  necessary  ingredient  for  efficient 
project  management,  it  has  also  long  been  correlated  with  risk  (because  of  activity 
interdependency),  and  might  vary  significantly  with  the  types  of  activities  underway  (See 
Figure  2) — the  inference  being  that  periods  of  stable  production  configuration  between 
development  increments  reduce  complexity  in  program  structure  and  attendant  risks. 
Similarly,  shorter  cycle-times  have  less  opportunity  for  knock-on  effects  or  secondary 
consequences.  Particularly  in  matrix  organization  structures,  as  often  the  case  with 
projects,  there  can  be  a  tendency  to  staff  multiple  projects  with  a  single  specialist.  The 
more  projects  a  specialist  supports,  the  less  they  are  proportionately  available  to  the 
projects  due  to  “queuing  inefficiencies. "Availability  decreases  because  of  the  need 
for  transition  between  projects  (physical,  mental,  learning  curve,  etc.). 

■  The  end  result  has  sometimes  been  shown  to  be  large  delays  in  project  completion 
(Smith  &  Reinhartsen,  1998). 

■  Similarly,  Ibrahim  (2005)  has  shown  that  discontinuous  enterprise  membership  is  a 
contributing  factor  toward  knowledge  loss  in  organizations  involved  in  large,  complex 
product  development  processes.  Examining  knowledge  flows  across  product 
lifecycles,  members  often  are  not  engaged  in  all  phases.  Whether  from  rotation  of 
duties  or  multi-tasking,  a  discontinuous  member’s  inaccurate  knowledge  could  cause 
a  functional  error  at  the  individual  level  which  is  not  obvious  at  the  enterprise’s 
overall  project  level.  These  findings  support  observations  of  knowledge  loss 
continuing  despite  investments  in  information  technology  and  knowledge 
management. 
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Development  Spiral s/lncrements  Concurrent  with  Initial  Production 


Development  SpIralsJlncrements  Concurrent  wrfth  Later  Production 


Figure  2.  Concurrency  Relative  to  Types  of  Activity 

•  User  Risk  (Safety  and  Time  Criticality).  Time  criticality  and  life-saving  dependency,  as 
opposed  to  user  hazard  levels  (safety  &  man-rating),  might  seem  to  also  have  influence 
over  design  approaches.  We  have  discussed  above  the  area  of  technological  risk  and 
the  DoD’s  use  of  incremental  or  spiral  approaches  to  resolve  it  (along  with  a  compulsory 
policy  for  the  advanced  development  of  only  relatively  mature  technology).  But  DoD 
products  have  expanded  risk  considerations  beyond  Boehm’s  models  of  commercial 
software.  Extending  the  idea  of  project  risk-as-a-driver  down  to  the  level  of  the  end-user, 
it  might  seem  logical  to  assume  that  time  criticality  of  the  need  or  mission,  where  risk  of 
not  achieving  project  success  actually  endangers  customer  lives,  might  be  a  significant 
factor  in  the  appropriate  application  of  the  spiral  process  for  reduced  initial  product  cycle¬ 
time.  Perhaps  defensive  systems  are  a  good  example.  The  immediate  needs  for  a 
Rocket-Propelled  Grenade  (RPG)  defeater  or  an  Improvised  Explosive  Device  (IED) 
neutralizer  for  currently  deployed  forces  in  Iraq  and  Afghanistan,  for  example,  clearly 
dictate  that  lives  will  be  lost  if  a  near-term  capability  is  not  achieved.  We  also  cite  as  an 
example  the  National  Missile  Defense  (NMD)  initiative,  in  which,  in  view  of  near-term 
threats,  early  deployment  of  even  rudimentary  capability  has  been  deemed  preferable  to 
waiting  for  full  capability.  Such  urgency  likely  precludes  full  and  certain  requirements 
specificity. 

■  Non-man-rated  Systems:  In  an  almost  opposite  vein,  non-man-rated  systems,  such 
as  Unmanned  Aerial  Vehicles  or  cave-exploring  robots — capabilities  in  which 
operator  lives  are  not  at  risk  if  the  product  fails — may  also  lend  themselves  readily  to 
rapid  innovation  and  risk-less  experimentation  cycles.  However,  user  hazard  levels 
for  man-rated  systems  may  be  a  different  matter. 
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■  Man-rated  Systems:  Configuration  variety  adds  technical  complexity  with 

unpredictable  interactions.  In  such  projects  as  pharmaceuticals,  aviation,  vehicular 
transportation,  etc.,  producers  mitigate  safety  risks  with  thorough  analyses, 
documentation  reviews,  testing  and  other  control  and  verification  processes.  By  their 
very  nature — with  lethal  hazards  for  the  end-user,  and  typically  lengthy  approval 
requirements — these  may  not  be  good  candidates  for  a  spiral  approach.  We  believe 
this  is  why  space  experts  say  they’ll  not  use  spiral  development  with  NASA’s  new 
Crew  Exploration  Vehicle  (Roy,  2006). 

•  Production  Quantity.  As  to  product  size  or  production  quantity,  we  find  no  evidence  that 
either  precludes  use  of  spiral — as  with  space  vehicles  and  large  ships — though  support 
considerations  do  arise  with  variety  that  could  greatly  affect  total  costs  of  ownership. 

•  Logistical  Support  planned  during  Service/Shelf  Life.  Our  observations  warn  that  multiple 
configurations  of  hardware  products  do  come  at  a  cost  for  ownership.  Veterans  of  new 
system  deployments  across  the  force/fleet,  or  throughout  any  large  using  organization, 
know  the  difficulties  of  rolling  out  a  configuration  change.  Benefits  of  standardization 
have  long  been  offered  via  production  economies  of  scale,  commonality  of  parts  across 
platforms,  and  interoperability.  If  the  ultimate  goal  is  to  have  standardization  across  the 
DoD’s  force,  owning  multiple  configurations  of  a  system  (variety)  equates  to  added 
complexity  in  training  and  supply  support  of  the  item.  Neither  can  the  logistical 
maintenance  strategy  be  ignored:  whether  the  end-item  is  maintenance-intensive  (such 
as  tactical  vehicles)  or  maintenance-free — such  as  with  many  electronics  items  and 
munitions,  and  situations  in  which  physical  changes  are  completely  transparent  to  the 
user.  For  multiple  product  configurations,  the  answer  could  have  a  huge  effect  on  the 
total  costs  of  ownership,  as  shown  by  RAND  on  the  proliferation  of  UAVs  (Shaver  & 
Amouzegar,  2005). 

•  Range  of  Reguirement  Attainment.  Certain  requirements  are  binary  rather  than 
continuous.  Examples  are  soft  launch,  network  security,  physical  fit,  leak-proof, 
shock/vibration/drop  proof,  survivability,  horizontal-to-vertical  flight  transition,  etc.  If  one 
of  these  more  binary-type  requirements  happens  to  be  a  key  performance  parameter,  its 
attainment  will  be  on  the  project’s  critical  path  and  highly  dependent  upon  technical 
maturity.  As  such,  it  may  practically  dictate  the  length  of  the  entire  advanced 
development  effort  and  make  division  into  capability  increments  less  beneficial  as  a 
development  strategy. 

•  Amount  of  Change — and  the  Lure  of  Modularity.  These  authors  subscribe  to  the  current 
theorists’  view  that  system  complexity  is  comprised  of  numbers  (of  components), 
connections  (interdependencies)  and  distinctions  (variety).  Distinction  corresponds  to 
variety,  to  heterogeneity,  and  to  the  fact  that  different  parts  of  complex  systems  behave 
differently  (Heylighen,  1997).  Variety  is  a  component  of  Nobel  Prize  winner  Herbert 
Simon’s  explanation  of  complexity — many  different  parts  with  many  interactions.  Simon 
argues,  from  his  observation  of  complexity  in  things  both  natural  and  artificial,  that 
complex  systems  evolve  from  simple  systems.  And  they  do  so  more  rapidly  when  there 
are  stable,  intermediate  forms  or  sub-systems  (like  modules  or  “units  of  action”)  (Simon, 
1981).  While  the  concept  of  modularity  suggests  approximately  independent  subsystems 
may  be  modified  or  adapted  as  such,  it  has  been  shown  that,  in  the  aggregate,  there  is 
yet  quantifiable  modular  interdependency  that  affects  evolvability  (Watson  &  Pollack, 
2005).  In  other  words,  how  changes  in  the  state  of  one  module  affect  the  state  of 
another  is  relative  and  measurable.  Thus,  we  suggest  it  is  not  only  the  focus  upon 
structural  modularity  as  such,  and,  standard  interfaces,  that  enable  systems  evolution. 
Rather,  it  is  the  relative  interdependency  of  the  modules.  In  short,  PMs  need  to  be 
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mindful  of  the  degree  of  change  in  subsequent  spirals.  One  subsystem  is  likely  to  affect 
another  in  the  short-  or  long-run.  And  that  can  make  product  evolution  problematic.  As 
Norman  Augustine  once  said,  “No  change  is  a  small  change”;  independent  subsystems, 
even  redundant  ones,  aren’t  always  independent  (Augustine,  1997,  June). 

Development  Case  Studies 

One  of  the  most  recent  monographs  we  have  found  on  emerging  results  of  evolutionary 
acquisition  is  by  RAND — on  five  immature,  non-man-rated  space  systems.  Space  systems  are 
somewhat  different  (in  quantities,  space  environment,  front-end  investment,  and  extended 
technology  development  periods)  than  other  DoD  sytsems.  RAND  also  found  that  policy 
confusion  persists,  and  that  EA  added  program  complexity  and  uncertainty,  especially  with 
regard  to  budgeting.  Extending  their  findings  to  non-space  DoD  programs,  RAND  highlighted 
the  EA  challenges  of  programmatic  flux.  They  feel,  and  we  agree,  that  EA  presents  the 
opportunity  for  typical  PM  challenges  to  be  even  more  formidable. 

Two  missile  programs  were  used  as  case  studies  for  analysis  and  to  illustrate  planned 
and  unplanned  change.  ATACMS  used  incremental  and  spiral  strategies  for  product 
development.  The  program  skipped  its  technology  development  phase  by  employing  mature 
technologies  for  a  leap-ahead  capability  in  range.  It  arrived  on  budget  and  schedule,  with 
several  successive  variants,  pre-planned  and  unplanned.  One  instance  of  production  change 
caused  missile  failure  and  costly  refit  of  already  produced  missiles — underscoring  the  need  for 
more  thorough  design  specification  and  configuration  management  accountability. 

Javelin  used  the  single-step-to-full-capability  approach  to  product  development.  The 
program  embarked  upon  advanced  development  with  immature  technologies  in  several  critical 
areas,  causing  significant  cost  and  schedule  overruns.  It  also  has  had  subsequent  design 
changes  and  product  variety,  more  so  as  running  production  changes  than  as  product  variants. 

Synthesis  of  these  cases  conveys  that  as  an  approach  oriented  primarily  for  reduction  of 
product  cycle-time,  incremental  or  spiral  development  can  successfully  be  used  when 
developing  mature  technologies  first.  But  a  system’s  physical  properties  like  mutability,  along 
with  other  factors  such  as  time  criticality  (user  risk)  and  modular  interdependency,  will  drive 
spiral  development  applicability.  And  key  capabilities  may  in  fact  depend  upon  the  least  mature 
technologies  or  even  binary  requirements.  An  “open,”  or  at  least  elegant,  architecture  is  key  to 
forming  a  basis  for  independent  modular  variety;  and  thorough  design  specification  and 
configuration  management  accountability  is  essential  for  managing  the  complexity  of  multiple 
product  releases.  All  amorphous  spirals  eventually  become  defined  increments.  Other  well- 
known  programs  have  used  a  spiral  approach  over  their  long  product  life  spans,  but  often  have 
successive  phasing  of  their  development  increments. 

Computational  Modeling  of  Spiral  Development 

A  computational  experimentation  approach  to  investigating  evolutionary  acquisition 
projects  is  explored  below.  This  approach  integrates  theory  and  practice  in  a  computational  tool 
that  allows  controlled  experimentation  through  simulation.  The  current  work  reflects  project 
theory  (e.g.,  the  theory  of  constraints  and  work  flows),  product  development  theory  (e.g.,  rework 
impacts  and  work  dependencies),  and  management  (e.g.,  resource  management  and 
information  theory).  Practice  is  reflected  in  the  model  through  the  use  of  case  studies  to  build 
and  validate  the  model  structures  (as  described  in  the  literature  cited)  and  the  calibration  and 
testing  using  the  acquisition  projects  described  above.  A  computational  experimentation 
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approach  provides  many  advantages  over  pure  and  benefits  from  several  of  the  strengths  of 
both  laboratory  and  field  research.  Nissen  and  Buettner  (2004)  describe  and  discuss  the 
computational  experimentation  approach,  and  Dillard  and  Nissen  (2007)  describe  its  application 
to  investigating  acquisition  projects. 

The  system  dynamics  methodology  was  applied.  System  dynamics  uses  a 
computational  experimentation  approach  to  understanding  and  improving  dynamically  complex 
systems.  The  system  dynamics  perspective  focuses  on  the  roles  of  accumulations  and  flows, 
feedback,  and  nonlinear  relationships  in  managerial  control.  The  methodology’s  ability  to  model 
many  diverse  system  components  (e.g.,  work,  people,  money),  processes  (e.g.,  design, 
technology  development,  quality  assurance),  and  managerial  decision-making  and  actions  (e.g., 
forecasting,  resource  allocation)  make  it  useful  for  investigating  acquisition  projects.  Forrester 
(1961)  develops  the  methodology's  philosophy,  and  Sterman  (2000)  specifies  the  modeling 
process  with  examples  and  describes  numerous  applications. 

Modeling  Incremental  Development  with  Multiple  Development 
Blocks 

Figure  3  depicts  an  acquisition  project  with  multiple  increments  or  blocks.  Subsequent 
blocks  have  the  same  basic  information  flow,  but  can  also  be  delayed  by  the  completion  of 
phases  in  previous  blocks  or  constrained  by  the  progress  in  their  own  blocks.  Importantly,  in 
addition  to  the  flow  of  information  downstream  through  phases  (black  arrows  in  Figure  3), 
multiple  iteration  acquisition  also  provides  opportunities  for  information  to  flow  upstream,  such 
as  from  User  Product  Testing  in  an  earlier  iteration  to  Develop  Requirements  or  Advanced 
Development  in  a  subsequent  iteration  (red  vertical  arrows  in  Figure  3). 
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Figure  3.  Information  Flows  in  an  Incremental  Acquisition  Project 

In  the  model,  the  structure  of  each  block  is  the  same,  although  parameter  values  are 
varied  to  reflect  different  acquisition  projects  and  strategies.  For  example,  all  phases  include 
start-up  work  that  is  not  directly  applied  to  generating  development  products  (requirements, 
technologies,  component  designs,  or  products).  Each  phase  also  includes  the  requisite  review 
work  that  also  does  not  directly  generate  product.  This  is  consistent  with  GAO 
recommendations  to  manage  each  development  block  like  an  individual  project.  One  impact  of 
this  loading  of  each  phase  with  start-up  and  review  work  that  we  suspect  has  only  been 
recognized  informally  is  a  significant  increase  in  the  total  amount  of  work  required  to  provide  a 
given  set  of  requirements  to  warfighters  when  multiple  development  blocks  are  used.  As  was 
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shown  with  our  modeling  results,  this  work  has  a  significant  impact  on  project  performance  that 
may  impact  the  types  of  projects  in  which  spiral  development  can  be  effective. 

Computational  modeling  of  incremental/spiral  versus  a  single-step  methodology  yields 
results  that  illustrate  our  implementation  concerns.  Spiral  development  can  provide  the  initial 
increment  delivery  with  some  (but  not  all)  requirements  satisfied  earlier  than  in  single-block 
development.  However,  spiral  development  takes  more  time  and  costs  more  to  satisfy  all 
requirements  than  single-block  development.  Spiral  development  has  a  high  risk  of  not 
satisfying  all  requirements  by  the  time  single-block  development  can  satisfy  all  requirements 
(See  Table  1). 
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Base  Case  - 
spiral 

Duration  to  first 
£  requirement  satisfied 

weeks 

471 

470 

397 

Duration  to  max. 

^  requirements  satisfied 

weeks 

520 

518 

762 

<u 

o  Total  development  cost 
as 

$1,000,000 

722 

719 

1,555 

§  Requirements  satisfied 
£  by  deadline 

% 

100 

91 

18 

Q-  Final  requirements 
satisfied 

% 

100 

91 

91 

Table  1.  Performance  Comparison  of  Three  Simulated  Acquisition  Projects 

The  causal  paths  that  drive  and  constrain  project  performance  in  spiral  development 
pass  through  multiple  types  of  resources,  development  processes,  and  move  across  both 
development  phases  and  development  blocks.  They  also  vary  widely  for  different  performance 
measures.  This  makes  the  drivers  of  and  constraints  on  spiral  acquisition  project  performance 
more  difficult  to  identify  than  those  in  single-block  development  projects.  Our  modeling  results 
indicate  that  spiral  development  is  a  significantly  different  approach  to  acquisition  than  single¬ 
block  development,  and  requires  different  planning,  resourcing,  and  management. 

The  concurrent  use  of  multiple  development  blocks  in  spiral  development  significantly 
increases  the  number  of  development  phases  and  activities  that  must  be  managed  and 
coordinated  at  any  given  time  compared  to  single-block  development.  This  increases  the  project 
management  needs  for  successful  acquisition  in  spiral  development  projects  compared  to 
single-block  projects. 

As  in  single-block  development,  progress  in  spiral  development  requires  the 
identification  and  understanding  of  progress  bottlenecks.  The  concurrence  and  resulting 
complexity  of  development  in  spiral  projects  causes  the  types  and  locations  of  bottlenecks  to 
vary  widely  and  be  more  difficult  to  identify  and  address  than  in  single-block  development. 
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Causal  paths  of  the  drivers  and  constraints  on  project  performance  and  progress 
bottlenecks  move  from  one  feature  of  a  project  to  another  as  projects  evolve.  The  increased 
dynamics  of  development  in  spiral  development  projects,  as  compared  to  single-block 
development,  make  identifying  and  addressing  causal  paths  and  progress  bottlenecks  more 
difficult.  Progress  bottlenecks  can  cause  counterintuitive  behavior,  such  as  reductions  in  project 
cost,  by  adding  resources  at  a  bottleneck.  Understanding  and  exploiting  the  opportunities 
provided  by  these  behaviors  requires  a  deep  understanding  of  the  project  structures  and 
dynamic  interactions  that  drive  and  constrain  progress. 

Discussion — Recent  Views  on  Balancing  Risk  and  Control 

Boehm’s  latest  book  on  software  development  advocates  balancing  disciplined  and  agile 
methods  to  capitalize  on  the  benefits  of  both.  Discipline  is  needed  as  a  control  mechanism  to 
avoid  risk,  but  agility  is  needed  to  respond  quickly  to  customer  needs.  Saying,  “One  size  fits  all 
is  a  myth”;  he  advocates  a  balanced  approach  based  upon  risk.  He  also  advocates  the  more 
disciplined,  risk-averse  approaches  for  projects  that  are  mission/safety  critical,  larger  in  size, 
and  have  more  stable  requirements  (Boehm,  2004). 

It  could  be  summarized  that  spiral  development  was  at  its  inception  and  is  at  its 
extension  all  about  risk.  Paradoxically,  it  is  an  agile  method  envisioned  to  reduce  risk  and,  yet, 
can  potentially  add  its  own.  On  the  one  hand,  a  spiral  or  incremental  approach  allays  risk  by 
reducing  scope  to  render  only  the  highest  priority  capabilities  with  the  exclusive  use  of  mature 
technology,  and  obtains  early  and  continuous  feedback  from  the  environment  for  follow-on 
developments.  On  the  other  hand,  it  introduces  concurrency  during  advanced  development  and 
adds  variety  in  production,  with  all  their  attendant  management  challenges. 

Observations  and  Assessments 

Although  today’s  policy  of  evolutionary  acquisition  is  prescribed  as  a  development 
methodology,  it  is  actually  focused  more  upon  what — not  how — we  develop.  As  such,  it  is  about 
doable  scope,  reducing  risk  via  exclusive  use  of  mature  technology.  The  Cost  As  an 
Independent  Variable  and  other  requirement-limiting  initiatives  were  earlier  attempts  to 
accomplish  this,  by  encouraging  product  performance  trades  to  keep  cost  estimates  fixed.  As 
with  CAIV,  this  likely  means  trading  performance  requirements  for  earliest-deploying 
increments. 

Spiral  development  also  seeks  to  spread  out  the  technical  risk  over  more  development 
and  process  time  via  incrementing.  We  have  shown  with  simulation  that  this  can  potentially 
improve  risk  management  performance  initially,  but  with  higher  overall  costs  and  longer 
subsequent  development  durations,  if  deliberately  deferring  known,  estimable  work.  As  such, 
our  computational  modeling  indicates  that  incremental  development  costs  more  and  requires 
more  time  to  provide  the  same  requirements  than  single-step  development.  With  regard  to 
project  risk,  the  increased  complexity  in  a  project  using  an  incremental  or  spiral  approach 
makes  the  isolation  and  effective  management  of  progress  bottlenecks  more  difficult  than  in 
single-step  development. 

The  policy  change  is  that  spiral  development  now  includes  undefinitized  increments  and 
prescribes  incremental  development  instead  of  single  step  development.  All  amorphous  spirals 
will  eventually  become  defined  increments — mini-programs.  In  years  past,  they  have  often  been 
implemented  as  sequential,  separate,  and  successive  product  upgrades  (such  as  the  CH-47, 
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UH-60,  C-130,  B-52  program  examples).  But  current  policy  expresses  these  as  more 
concurrent,  frequent  and  continuous.  Such  concurrency  adds  complexity  to  development 
models,  with  attendant  risks  of  over  allocation  of  work,  noise,  error,  duplication,  and  other 
inefficiencies  from  work  deferral  and  divided  effort  in  project  management  organizations. 
Additional  oversight,  reviews,  contracting,  testing,  etc.,  will  also  likely  affect  transaction  costs.  If 
all  requirements  are  known  and  an  incremental  approach  is  used,  then  there  is  a  deliberate 
deferral  of  work  to  later  increments,  and  there  will  be  a  resultant  increase  in  total  development 
costs  and  durations  for  these  same  reasons. 

Recommendations  for  Practice 

1 .  Project  managers  need  to  be  aware  of  the  inherent  risks  of  spiral  development  and  take 
necessary  precautions  to  balance  those  risks.  Many  tools  and  control  measures  are 
currently  developed  and  available  to  assist  project  managers  in  balancing  the  risks  of 
spiral  development,  such  as  technology  readiness  levels,  configuration  management, 
technology  performance  management,  real  options,  project  phasing,  risk  management, 
earned  value  management  and  organizational  design. 

2.  Incremental  and  spiral  development  projects  provide  additional  opportunities  for 
managing  development  risk  in  the  project  design.  These  include  project-planning 
decisions  about  the  number  of  development  blocks,  the  requirements  and  associated 
technologies  and  design  components  to  be  included  in  specific  blocks.  This  planning 
provides  opportunities  to  anticipate  where  critical  progress  bottlenecks  may  occur  and 
design  how  to  best  monitor  potential  bottlenecks  and  respond  to  them. 

3.  Product  attributes  may  help  determine  the  suitability  of  spiral  development  as  the  best 
methodology.  PMs  may  wish  to  consider  such  characteristics  as:  mutability,  time 
criticality,  man-rating,  modular  interdependency,  key  parameters  of  capability  versus 
range  of  requirement  attainment  (i.e.,  binary  vs  continuous),  and  the  relative  amount  of 
concurrency  among  increments. 

4.  Progress  bottlenecks  in  iterative  and  spiral  development  often  oscillate  between  process 
constraints  (e.g.,  availability  of  work  due  to  upstream  progress)  and  resource  constraints 
(developer  or  project  management  quantities  or  productivities).  Successfully  addressing 
a  constraining  progress  bottleneck  often  shifts  the  limit  on  progress  to  a  different  location 
in  the  project.  Therefore,  a  structured  and  interdisciplinary  practice  of  identifying  and 
addressing  bottlenecks  can  improve  performance. 

5.  Configuration  management  accountability  must  be  assigned  or  kept  to  maintain 
supportability  and  failure-mode  identification  and  causality  and  prevent  the  variety 
generated  by  spiral  development  from  reducing  total  product  performance. 

Conclusions 

We’ve  suggested  that  a  one-size-fits-all  methodology  for  DoD  system  development  may 
not  be  appropriate,  and  have  offered  for  consideration  several  product  attributes  that  might  help 
determine  the  efficacy  of  the  spiral  approach.  We  further  suggest  that  spiral  development  may 
serve  better  than  single-step  development  for  initial  capability  when  products  are  mutable,  time- 
critical,  non-maintenance  intensive,  and  have  continuous  (vs.  binary)  or  uncertain  requirements, 
short  cycle-times  (less  knock-on  effects),  sequentially  phased  development,  and  modular 
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independence.  In  contrast,  spiral  development  may  not  be  appropriate  when  there  are  safety  or 
man-rating  concerns  and  have  attributes  opposite  to  those  above.  In  particular,  PMs  should 
understand  the  nature  of  their  product  requirements  with  regard  to  their  range  of  attainment  and 
relative  to  key  parameters  of  capability,  and  vis-a-vis  the  readiness  level  of  their  enabling 
technologies.  Some  key  features  may  indeed  be  binary,  and  others  may  have  significant 
ramifications  of  partial  attainment — such  as  propagated  change  across  the  entire  product 
componentry  (as  in  weight  reduction),  versus  a  more  independent,  modular  modification. 

Open  design  standards  will  not  always  be  incorporable.  And  product  variety  will  emerge, 
with  and  without  backward  compatibility,  interoperability,  etc.  Variety  is  both  an  asset  (for  end- 
users)  and  a  liability  (for  manufacturers,  owners  and  supporters).  As  such,  to  compensate  for 
product  variety,  “owners”  must  “own”  the  design  and  emphasize  configuration  management, 
keeping  or  assigning  responsibility  for  that  function,  and  maintaining  accountability  for  it. 

Both  product  specifications  as  well  as  risk  realization  in  spiral  development  move  from 
being  amorphous  to  defined.  Spiral  development  has  inherent  challenges,  both  strategic  and 
tactical,  of  which  PMs  must  be  aware.  We’ve  highlighted  and  illustrated  them  here,  as  well  as 
showing  (in  our  case  studies)  that  spiral  development  can  indeed  work — especially  for 
technically  mature  and  mutable  products  with  open  or  elegant  architecture.  Program  Managers 
must  be  aware  of  these  inherent  risks,  and  take  necessary  precautions  to  balance  them  with 
increased  use  of  tools,  such  as  technology  readiness  levels,  configuration  management, 
technical  performance  measurement,  contract  incentives,  options  and  phasing,  organizational 
design,  etc. 

Stability  is  the  quest  in  all  things  programmatic — for  funding,  requirements,  design, 
configuration,  etc.  But  in  an  unstable  world,  and  with  the  future  being  necessarily  uncertain,  the 
tension  between  control  and  change  is  probably  unending.  PMs  do  have  some  tools  for  coping, 
and  being  forewarned  is  being  forearmed.  PMs  are  used  to  concurrency  and  change,  as  they 
are  largely  what  make  project  management  what  it  is — a  balancing  act.  Mechanisms  for  control 
of  risk  include  project  management  tools  such  as  configuration  management,  technical 
performance  measurement,  earned  value  management,  risk  management,  etc.  Organizational 
and  cultural  factors  such  as  leadership,  trust  and  accountability  play  a  significant  role  as  well. 
Successful  use  of  these  tools  to  balance  control  and  risk  in  projects  with  a  high  rate  of  change 
and  concurrency  is  an  area  for  our  further  study. 
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I.  Introduction 

There  is  mounting  evidence  of  a  systematic  bias  in  initial  cost  estimates  of  new  weapon 
systems.  A  comprehensive  2006  RAND  report  on  major  weapons  programs  concludes:  “[The] 
analysis  indicates  a  systematic  bias  toward  underestimating  the  costs  [...]  of  a  weapon  system.” 
(Arena  2006,  p.1).  This  bias  could  threaten  our  national  security.  Unrealistically  low  cost 
estimates  result  in  cost  overruns.  Fixing  cost  overruns  can  impact  military  readiness. 

Two  factors  are  usually  blamed  for  unrealistically  low  cost  forecasts:  bad  incentives 
(psychological  and  political-economic  explanations)  and  bad  estimates  (methodological 
explanations).  The  focus  of  this  study  is  on  cost  methodology.  Our  goal  is  to  contribute  some 
new  ideas  to  the  current  literature  on  cost  estimating.  This  paper  applies  Transaction  Cost 
Economics  (TCE)  (e.g.,  Williamson,  1985;  Dillard,  Franck  &  Melese,  2006)  to  help  characterize, 
explain,  and  ultimately  reduce  the  cost  growth  that  plagues  many  of  today’s  major  investments 
in  military  capabilities. 

In  business,  two  costs  are  typically  factored  into  the  “make-or-buy”  decision:  production 
costs  and  the  costs  of  managing  transactions — “transaction  costs”  (Coase  1937).  Conventional 
estimation  techniques  tend  to  focus  on  production  costs  (input  costs,  learning  curves, 
economies  of  scale  and  scope,  etc.).  TCE  emphasizes  another  set  of  costs — primarily  the  costs 
of  coordination  and  motivation  (e.g.,  search  and  information  costs,  decision  and  contracting 
costs,  monitoring  and  enforcement  costs).  The  primary  insight  drawn  from  TCE  is  that  correctly 
estimating  the  economic  production  costs  of  an  acquisition  is  necessary,  but  not  sufficient.  The 
choice  of  contract,  organization,  and  incentives,  along  with  key  characteristics  of  markets  and 
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transactions  (uncertainty,  complexity,  asset  specificity,  frequency,  and  contestability),  must  be 
included  to  obtain  reliable  cost  estimates. 

II.  Background 

A  recent  Government  Accountability  Office  (GAO)  study  cites  multiple  examples  of 
weapon  systems  cost  increases.  The  GAO  reports  significant  unit  procurement  cost  increases 
for  several  familiar  programs.  The  Joint  Strike  Fighter  saw  unit  procurement  costs  increase  over 
25%,  the  Army’s  Future  Combat  Systems  over  50%,  while  the  Air  Force’s  F-22A  Raptor 
experienced  almost  200%  procurement  cost  overruns  (GAO  April  2006,  p.5).  According  to 
another  GAO  report,  “[programs  consistently  move  forward  with  unrealistic  cost  [...]  estimates” 
(GAO  Mar  2006,  p.1). 

Traditional  cost  estimating  relies  on  a  Work  Breakdown  Structure  (WBS).  Similar  to  an 
economist’s  production  function,  a  WBS  captures  the  mix  of  inputs  and  activities  required  to 
produce  a  specific  weapon  system.  Cost  estimates  developed  for  each  component  of  the  WBS 
are  rolled  up  into  an  overall  estimate  for  the  project.  Also,  simulation  methods  can  be  used  to 
capture  uncertainty  and  provide  a  distribution  of  possible  weapon  system  costs.  Confidence 
intervals  applied  to  this  distribution  allow  cost  estimators  to  report  probabilities  associated  with 
specific  ranges  of  costs  for  a  particular  weapon  system.  However,  the  confidence  in  these  cost 
estimates  depends  inter  alia  on  the  completeness  of  the  model.  TCE  offers  complementary 
considerations  that  can  impact  the  costs  of  major  weapon  systems,  many  of  which  are  known 
but  not  explicitly  factored  into  traditional  cost  estimates. 

In  its  recent  review  of  the  literature,  a  RAND  study  reports,  “our  analysis  [...]  shows  that, 
by  and  large,  the  DoD  and  military  departments  have  underestimated  the  cost  of  buying  new 
weapon  systems”  (Arena,  2006).  Virtually  all  the  studies  that  RAND  examined  found  a 
systematic  downward  bias  in  cost  estimates.  The  average  cost  growth  over  initial  forecasts  of 
weapon  systems  in  the  development  phase  ranged  from  16  to  26%.  Estimates  of  procurement 
cost  growth  averaged  between  16%  and  65%,  while  total  weapon  program  cost  overruns 
averaged  between  20  to  54%. 

Cost  estimates  serve  two  main  functions.  First,  they  serve  as  an  integral  part  of  the 
decision  process  (cost-benefit  or  “analysis  of  alternatives”)  used  to  evaluate  military 
investments.  Second,  they  provide  the  foundation  for  future  defense  budgets. 

In  the  first  case,  underestimating  costs  can  result  in  overestimates  of  the  affordable 
quantity  of  those  weapons.  It  can,  therefore,  result  in  too  many  new  weapon  program  starts  and 
excessive  investments  in  those  systems.  In  the  second  and  related  case,  unrealistically  low  cost 
estimates  result  in  overly  optimistic  budgets.  Budgets  planned  on  the  basis  of  optimistic  cost 
estimates  create  the  illusion  of  more  resources  available  than  actually  exist.  Since  acquisition 
budgets  are,  in  essence,  a  contract  for  major  weapon  systems  between  the  DoD  and  Congress, 
unrealistic  cost  estimates  can  lead  to  a  breach  of  trust  and  can  poison  relations  between  the 
branches  of  government. 

One  method  used  to  cope  with  cost  overruns  is  to  stretch  out  programs  and  cut 
quantities  that  reach  the  force.  But  this  sacrifices  both  current  and  future  operational  capability. 
Moreover,  spreading  fixed  costs  over  fewer  units  increases  unit  production  costs.  The  end  result 
can  be  to  get  less  (quantity)  for  more  (money).  Another  method  to  pay  for  cost  overruns  is 
finding  savings  in  training,  operations,  and  maintenance  budgets.  But  this  also  risks  sacrificing 
current  and  future  operational  capability.  Reprogramming  money  from  other  programs  to 
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accommodate  cost  increases  is  another  approach.  But  this  is  a  limited  remedy  (constrained  to 
no  more  than  $10M  in  RDT&E  and  $20M  in  Procurement  within  a  program  element)  and  can 
have  a  negative  and  cascading  effect  on  other  programs’  costs,  schedules,  and  performance. 
The  net  effect  of  cost  overruns  is  clear.  Short  of  a  bailout  from  Congress,  systematically 
underestimating  costs  reduces  the  overall  readiness  and  availability  of  military  forces  and 
equipment. 

III.  Alternative  Explanations  of  Unrealistic  Cost  Estimates 

The  two  factors  commonly  used  to  explain  “unrealistic  cost  estimates”  are  bad  incentives 
[due  to  optimism  bias  (psychological  explanation),  or  perverse  bureaucratic  incentives  and 
strategic  misrepresentation  of  budgets  (political-economic  explanation)],  and  bad  estimates 
[imperfect  forecasting  techniques  such  as  an  omitted  variable  bias  (methodological 
explanation)].  It  is  helpful  to  examine  these  factors  in  the  context  of  the  Planning,  Programming, 
Budgeting  and  Execution  (PPBE)  process  used  by  the  DoD  to  build  the  nation’s  defense 
budget. 

In  theory,  “Programming”  involves  a  constrained  optimization  in  which  investment  and 
operating  decisions  are  made  to  maximize  national  security  subject  to  fiscal  constraints.  The 
Secretary  of  Defense  (including  OSD  Program  Analysis  and  Evaluation  (PA&E)  and  the  DoD 
Comptroller)  and  the  Chairman  of  the  Joint  Chiefs  of  Staff  have  good  reasons  to  prefer  accurate 
cost  estimates  given  their  global  optimization  perspective. 

In  contrast,  given  their  sub-optimizing  perspectives,  Combatant  Commanders  and  the 
Military  Services  (for  instance)  may  be  more  subject  to  a  bias  toward  optimism  and  to  perverse 
bureaucratic  incentives.  Since  their  primary  focus  is  on  performance  and  military  capability,  they 
have  reason  to  be  less  critical  (and  realistic)  in  their  cost  estimates. 

According  to  McNicol  (2005,  p.  S-4),  “statistical  analysis  is  consistent  with  the  well- 
established  presumption  that  the  military  services  tend  to  prefer  [...]  optimistic  procurement  cost 
estimates.”  Program  Element  Monitors  (PEMs),  for  example,  act  as  advocates  for  the  funding  of 
their  programs  and  are  responsible  for  defending  those  programs  throughout  the  PPBE  cycle. 
Since  all  programs  compete  for  the  DoD’s  limited  resources  in  a  particular  funding  categories, 
PEMs  are  well  aware  that  resource  allocation  is  a  competitive,  constant-sum  game.  Given  their 
job  descriptions,  PEMs  are  drawn  to  lower  cost  estimates  that  fit  their  advocacy  agendas,  rather 
than  to  higher  cost  estimates  which  make  defending  their  programs  more  difficult.  Moreover, 
since  they  do  not  always  posses  a  higher-level  perspective,  they  may  not  fully  appreciate  how 
underestimating  costs  can  impact  other  programs. 

Also,  defense  companies  have  good  reasons  to  strategically  underestimate  costs  since 
this  makes  their  programs  more  attractive  to  the  DoD  and  the  Congress.  Here,  the  logic  can  be 
interpreted  by  leveraging  one  of  the  hallmarks  of  TCE,  the  “fundamental  transformation” 
(Williamson,  1996).  The  standard  example  is  when  ex  ante  competitive  bidding  leads  to  an  ex 
post  bilateral  monopoly  situation.  The  risk  is  that  the  winning  supplier  can  lock  in  the 
Government  by  making  investments  in  productive  assets  that  are  specific  to  the  relationship 
(and  that  have  little  value  outside  the  relationship).  Once  the  DoD  “buys  in,”  defense  firms  can 
lock  in  their  customer  by  investing  in  specific  assets  (e.g.,  production  facilities)  that  act  as 
barriers  to  entry.  While  initially  advantageous,  such  investments  in  specific  assets  can  make  it 
prohibitively  costly  for  other  companies  to  compete  in  subsequent  re-bidding  of  the  contract. 
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Given  the  complexity  and  uncertainty  of  many  major,  technologically  advanced  weapon 
systems,  contracts  are  necessarily  incomplete;  i.e.,  they  cannot  cover  all  possible 
contingencies.  As  a  consequence,  the  incumbent  company  can  be  confident  it  will  recoup 
overruns  from  multiple  change-orders  (generally  for  added  scope)  anticipated  over  the  life  of  the 
contract.  In  fact,  firms  have  an  incentive  to  anticipate,  but  strategically  omit,  some  of  these 
elements  in  the  original  contract  negotiation.  Moreover,  by  strategically  hiring  workers  in  key 
Congressional  districts,  the  company  can  pressure  Congress  to  retain  the  contract  and  approve 
compensation  for  cost  overruns  to  preserve  jobs.  This  combination  of  strategic  behaviors  could 
explain  some  of  the  systematic  bias  in  initial  cost  estimates. 

Therefore,  cost-estimating  techniques  must  properly  anticipate  transaction  costs  such  as 
measurement,  monitoring,  management  and  re-negotiation  costs  that  can  quickly  overwhelm 
initial  production  cost  estimates,  and  which  vary  according  to  the  nature  of  the  transaction.  If 
TCE  considerations  are  not  properly  anticipated,  the  result  will  be  that  cost-estimators  will 
continue  their  downward  bias,  and  programs  will  continue  to  suffer  cost  overruns.  One  possible 
means  to  improve  DoD  cost  estimation  is  to  add  transaction  cost  considerations  to  the  current 
production  cost  focus  in  cost-estimating  methods. 

In  an  early  attempt  to  address  the  factors  that  lead  to  biased  cost  estimates,  Secretary  of 
Defense  Melvin  Laird  directed  that  independent  parametric  cost  estimating  be  made  a  part  of 
the  DoD  acquisition  process.  The  Office  of  the  Secretary  of  Defense  (OSD)  launched  an 
independent  Cost  Analysis  Improvement  Group  (CAIG)  that  began  operations  in  the  Spring  of 
1972.  Eventually,  the  CAIG  was  assigned  the  statutory  responsibility  of  providing  independent 
lifecycle  cost  estimates  for  all  major  weapon  acquisition  programs.  According  to  McNicol  (2005), 
“the  introduction  of  independent  parametric  costing  [...]  had  a  major,  continuing  effect  on 
reducing  procurement  cost  growth”  (p.44). 

Whereas  independent  cost  estimation  is  an  important  step  that  clearly  attenuates  the 
impact  of  psychological  and  political-economic  factors,  the  challenge  of  developing  a  more 
comprehensive  cost  estimating/forecasting  methodology  remains.  Hence,  the  primary  focus  of 
this  study  is  to  leverage  insights  from  TCE  to  improve  the  DoD’s  cost  estimating/forecasting 
methodology — while  recognizing  those  other  causes  of  bias  in  cost  estimates. 

The  next  section  (IV)  reviews  traditional  cost  estimating.  The  following  section  (V) 
reviews  some  key  contributions  of  Transaction  Cost  Economics.  Section  VI  introduces  two  sets 
of  hypotheses  relating  to  DoD  cost  estimating  and  describes  how  insights  from  TCE  might  be 
leveraged  to  improve  DoD  cost  estimation.  The  following  section  (VII)  of  the  paper  offers  two 
case  studies  interpreted  in  the  context  of  TCE.  Section  VIII  reports  our  search  for  evidence  of 
transaction  costs  found  within  program  office  budgets  and  its  results.  The  concluding  section 
(IX)  offers  some  policy  recommendations  to  improve  future  DoD  cost  estimating/forecasting. 
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IV.  A  Brief  Review  of  Cost  Estimating  in  the  Department  of 
Defense 

In  the  defense  acquisition  process,  a  cost  estimate  is  a  prediction  or  forecast  of  the  cost 
of  a  program  or  weapon  system.  DoD  Instruction  5000.2  requires  that  both  a  program  office 
estimate  and  a  DoD  component  cost-analysis  estimate  be  prepared  in  support  of  acquisition 
milestone  reviews.  For  major  defense  acquisition  programs,  these  estimates  are  subject  to 
review  by  the  OSD  Cost  Analysis  Improvement  Group  (CAIG).  In  addition,  the  CAIG  performs 
an  independent  lifecycle  cost  estimate  for  all  ACAT  ID  programs  and  for  certain  ACAT  1C 
programs.13 

The  first  step  in  any  cost  estimate  is  to  understand  the  attributes  of  the  program  or 
weapon  system  whose  cost  is  to  be  estimated.  Traditionally,  this  requires  understanding  and 
describing  the  weapon  system  in  terms  of  physical  and  technical  parameters,  operational  and 
support  concepts,  mission  requirements,  and  interfaces  with  other  systems.  Understanding  the 
program’s  schedule  and  acquisition  profile  is  also  important  in  developing  a  cost  estimate.  The 
goal  is  to  understand  the  relationship  between  key  weapon  system  attributes  and  cost.  For 
programs  reviewed  by  the  CAIG,  this  first  step  is  documented  in  the  Cost  Analysis 
Requirements  Description  (CARD),  according  to  the  guidelines  in  DoD  5000. 4-M. 

The  second  step  is  to  develop  an  explicit  framework  for  the  cost  estimate.  The  Work 
Breakdown  Structure  (WBS)  is  a  hierarchy  of  weapon  system  components  (hardware,  software, 
data,  facilities,  and  services)  that  attempts  to  capture  economic  production  cost  elements  of  the 
estimate.  A  WBS  is  developed  by  subdividing  a  product,  process  or  service  into  its  major  work 
elements  and  sub-elements.  The  highest  level,  the  system  WBS,  can  be  organized  according 
to  the  lifecycle  phases  of  a  system:  Research  &  Development,  Investment,  Operating  and 
Support  and  Disposal. 

Under  each  phase,  a  hierarchical  structure  documents  the  activities  and  resources 
required  to  complete  the  work  associated  with  that  phase.  Under  the  Investment  phase,  for 
example,  the  Program  WBS  encompasses  the  entire  acquisition  program,  including  production 
costs  derived  from  the  Contract  WBS.  This  defines  at  an  aggregated  level  what  is  to  be 
procured  and  consists  of  at  least  three  program  levels  with  associated  definitions. 

The  Program  WBS  is  used  by  the  Government  program  manager  and  contractor  to 
develop  and  extend  a  Contract  WBS.  It  contains  uniform  terminology,  definitions,  and 
placement  in  the  input-oriented  family  tree  structure.  The  contract  WBS  provides  the  structure 
for  information  contained  in  the  Contractor  Cost  Data  Reporting  System  and  other  cost 
performance  reports  and  is  defined  by  Military  Handbook  881  A.  Operating  and  Support  costs 
are  organized  in  a  Cost  Element  Structure  in  accordance  with  CAIG  guidance,  similar  to  the  way 
the  WBS  is  used  to  organize  development  or  production  costs.  An  example  of  a  program  WBS 
is  shown  in  Figure  1. 


13  Acquisition  Category  (ACAT)  I  is  defined  by  DODI  5000.2  as  a  major  defense  acquisition  program 
(MDAP)  that  either  has  a  dollar  value  estimated  by  the  USD(AT&L)  to  require  an  eventual  total 
expenditure  for  research,  development,  test  and  evaluation  (RDT&E)  of  more  than  $365  million  in  fiscal 
year  (FY)  2000  constant  dollars  or,  for  procurement,  of  more  than  $2,190  billion  in  FY  2000  constant 
dollars,  or  has  been  designated  of  special  interest  by  the  Milestone  Decision  Authority  (MDA).  (DODI 
5000.2,  enclosure  2) 
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The  level  of  detail  in  the  WBS  evolves  as  the  system  is  defined  and  developed 
throughout  its  lifecycle.  After  the  program  WBS  has  been  approved,  the  contractor  will  then 
extend  the  contract  WBS  to  the  appropriate  lower  levels  to  better  define  the  complete  scope  of 
the  contract.  When  integrated  with  the  program  WBS,  the  extended  contract  provides  a 
complete  WBS  for  the  acquisition  program.  Similar  to  the  economists’  production  function,  the 
WBS  reveals  all  the  key  inputs  required  to  generate  the  ultimate  output/product. 


Level  1 


Level  2 


Level  3 


Figure  1.  Program  WBS 

(DoD,  2005) 

The  next  step  is  to  develop  a  cost  estimate.  From  the  WBS  perspective,  the  total  cost  of 
the  system  is  obtained  by  adding  up  the  cost  of  the  individual  elements  (inputs)  in  the  hierarchy 
across  the  levels  of  the  WBS.  In  order  to  do  this,  a  cost  estimate  must  be  generated  for  each 
element  of  the  WBS.  The  level  of  detail  of  the  cost  estimate  depends  on  the  amount  of 
information  available  about  the  system.  In  the  early  stages  of  development,  there  are  only 
rough  estimates  of  the  system  parameters/attributes;  therefore,  only  the  highest  levels  of  the 
WBS  cost  can  be  estimated  using  “rough  order  of  magnitude”  or  “top-down”  techniques  such  as 
analogy  and  parametric  or  cost  estimating  relationships  (Blanchard,  p.  595).  For  example,  the 
estimated  cost  of  a  new  aircraft  may  be  modeled  as  a  function  of  the  weapon  system’s  key 
inputs/attributes  like  empty  weight,  speed,  useful  load,  wing  area,  power,  and  landing  speed. 
This  model  estimates  the  total  cost  of  the  aircraft  based  on  a  set  of  parameters  independent  of 
the  WBS  structure. 

As  more  information  about  the  system  becomes  available,  more  detailed  estimates  can 
be  developed  for  lower  levels  of  the  WBS  using  a  variety  of  techniques.  A  cost  estimate  is  now 
developed  for  each  element  at  a  given  level  of  the  WBS.  For  example,  the  cost  of  the  aircraft 
wing  may  be  estimated  using  historical  data  for  weight,  area,  and  materials.  While  this  estimate 
is  still  based  on  the  parametric  technique,  the  level  of  detail  (a  wing)  corresponds  to  a  WBS 
element.  As  the  system  matures,  the  work  and  resource  requirements  are  sufficiently  well 
defined  to  use  ’’bottoms  up”  techniques  such  as  engineering  estimates  and  actual  costs 
obtained  from  prototypes  and  early  production  models.  An  engineering  (production  function) 
estimate  would  be  based  on  the  labor,  material  and  overhead  required  to  complete  a  particular 
element  of  the  WBS.  Obviously,  this  sort  of  estimation  requires  a  great  deal  of  information 
about  how  the  system  is  built — data  that  is  usually  not  available  until  the  early  phases  of 
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production.  Figure  2  shows  the  relationship  between  estimating  techniques  and  the  evolution  of 
a  system. 


Figure  2.  Cost  Estimating  Techniques  as  a  Function  of  Acquisition  Phases 


While  the  WBS  framework  provides  an  excellent  accounting  system  for  cost  estimates,  it 
does  have  a  few  drawbacks.  First,  although  it  does  capture  the  functional  relationship  between 
inputs/elements,  it  does  not  explicitly  show  the  correlation  between  cost  elements.  If  there  is  a 
correlation  between  cost  elements  that  are  assumed  independent,  this  can  significantly  increase 
the  variability  of  the  cost  estimate.  Second,  the  program  WBS,  as  defined  by  Military  Handbook 
881 A  ,  is  input-oriented,  not  relationship-oriented;  it,  therefore,  largely  overlooks  transaction 
costs  such  as  search  and  information  costs,  decision  and  contracting  costs,  monitoring  and 
enforcement  costs.  While  it  might  be  argued  these  costs  are  implicitly  considered  in  the  cost 
estimates  of  the  various  WBS  components,  it  is  likely  that  most  are  either  underestimated  or 
ignored,  resulting  in  overly  optimistic  cost  estimates. 

V.  A  Brief  Review  of  Transaction  Cost  Economics  (TCE) 

Transaction  costs  are  typically  faced  by  organizations  dealing  with  outside  suppliers. 
These  include  costs  associated  with:  source  selection,  periodic  competition  and  renegotiation, 
contract  management,  market  structure,  and  measuring  and  monitoring  performance.  TCE 
emphasizes  four  key  characteristics  of  transactions:  complexity,  uncertainty,  frequency,  and 
asset  specificity.  Transaction  Cost  Economics  offers  an  attractive  theoretical  foundation  for 
competitive  sourcing  decisions  in  the  private  sector  (e.g.,  Coase,  1937;  Williamson,  1971; 

1979;  Alchian  &  Demsetz,  1972).  However,  TCE  has  been  less  often  applied  in  a  government 
setting  (e.g.,  Pint  &  Baldwin,  1997;  Williamson,  1999;  Franck  &  Melese,  2005;  Dillard,  Franck  & 
Melese,  2006).  An  underlying  objective  of  TCE  (as  a  field  of  inquiry)  is  to  improve  the  design  of 
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contracts,  organizations,  and  other  governance  structures  that  reduce  transaction  costs  and 
improve  the  gains  from  exchange  between  buyers  and  sellers. 

TCE  predicts  parties  involved  in  a  defense  transaction  can  benefit  from  cooperation  in 
the  buyer-seller  relationship  and,  thereby,  generate  a  surplus  that  can  be  shared.  For  example, 
specific  investments14  made  by  either  party  that  are  of  real  value  in  the  relationship  (but  perhaps 
of  lesser  value  outside  the  relationship),  can  improve  efficiency  and  effectiveness — yielding  a 
surplus  to  be  shared  between  the  two  parties.  However,  those  parties  often  have  conflicting 
interests;  the  DoD  wants  to  maximize  defense  capabilities  subject  to  budget  constraints  while 
Industry  must  concern  itself  with  profits.  Therefore,  both  sides  have  incentives  to  behave 
opportunistically,  and  may  not  necessarily  have  sufficient  motivation  to  make  investments  that 
increase  the  parties’  total  gains.  This  is  particularly  true  when  specific  assets  are  involved  and 
information  is  imperfect  and  asymmetric.  While  defense  acquisition  focuses  on  production  costs, 
it  also  exposes  the  organization  to  costs  of  managing  the  outsourcing  relationship  and  to  the 
risks  of  bad  (opportunistic)  behavior  on  the  part  of  contracting  partners. 

Relationship-specific  investments  are  potentially  valuable,  but  can  increase  risks  to  both 
parties  in  a  transaction.  Having  made  a  specialized  investment,  the  supplier  becomes  the  most 
efficient  provider,  and  thus  has  an  incentive  to  look  for  opportunities  to  extract  more  of  the 
surplus  (perhaps  by  demanding  large  increases  in  contract  price  to  execute  change  orders).  If  a 
customer  is  “locked  in,”  they  may  have  little  recourse.  At  some  point,  the  relationship  is 
transformed  from  a  customer  having  the  choice  of  a  number  of  competing  suppliers  to  a  bi¬ 
lateral  monopoly  relationship  between  a  single  buyer  and  single  seller.  At  this  point,  close-in 
bilateral  bargaining  replaces  the  impersonal  (arms’-length)  arrangements  of  the  competitive 
marketplace. 

This  entails  a  basic  transformation  of  the  supplier  from  competitive  bidder  (prior  to 
source  selection)  to  monopoly  supplier  (after  source  selection),  especially  if  there  are  no  close 
substitutes  for  this  particular  contractor’s  products.  Accordingly,  the  customer  is  now  vulnerable 
to  “opportunistic  behavior”15  from  the  contractor.  Unforeseen  circumstances  combined  with 
newly  inelastic  demand  may  prompt  the  supplier  to  extract  more  of  the  surplus  created  in  the 
relationship.16  The  supplier  can  exploit  its  power  in  the  relationship  to  renegotiate  the  basic 
agreement  to  its  advantage,  otherwise  threatening  to  dissolve  the  agreement.  The  TCE 
literature  refers  to  this  as  a  “hold  up.”17  This  is  one  of  the  key  insights  TCE  can  offer  to  improve 
initial  cost  estimating. 

If  the  supplier  makes  specific  investments  in  assets  that  are  only  valuable  in  the  context 
of  the  relationship  with  a  specific  customer,  it  is  vulnerable  to  any  changes  in  demand  from  that 
customer.  Whereas  relation-specific  investments  can  increase  the  total  gains  from  the  defense 


14  Asset  specificity  comes  in  a  variety  of  forms,  such  as  human,  location  and  physical.  These  are 
specialized  assets  that  generate  high  returns  within  a  specific  relationship,  but  offer  little  value  outside  it. 

15  Williamson,  1996,  defines  “opportunism”  as  “self-interest  seeking  with  guile” 

16  Besanko,  Williamson  and  others  have  labeled  the  transition  from  one  prospective  buyer  and  many 
sellers  to  one  buyer  and  one  seller,  from  competitive  market  to  a  one-on-one  relationship,  as  the 
“fundamental  transformation.”  This  transformation  occurs,  at  least  to  a  certain  extent,  after  the  completion 
of  every  source-selection  process. 

17  According  to  Besanko,  “a  hold  up  problem  arises  when  a  party  in  a  contractual  arrangement  exploits 
the  other  party’s  vulnerability  due  to  relationship-specific  assets.” 
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acquisition,  they  also  increase  risks  of  opportunistic  behavior,  through  which  either  party  can 
hold  up  the  other.  For  example,  either  party  can  hold  up  the  other  by  threatening  to  change  the 
terms  of  the  contract  (relationship).  The  danger  is  that  if  neither  party  feels  like  it  can  recover  the 
full  costs  of  its  investment  in  the  relationship/transaction  (say  through  a  continuation  or  renewal 
of  the  contract),  then  efficiency-generating  specific  investments  will  not  be  made,  resulting  in 
higher  (unanticipated)  costs.18 

A  crucial  insight  of  TCE  is  that  different  ex  ante  contracts  offer  different  incentives  for 
unproductive  ex  post  bargaining  and  influence  activities.  As  in  game  theory,  it  helps  to  look 
forward  and  reason  back.  If  it  appears  managing  the  contract  (including  future  competitions 
and/or  renegotiations),  and  evaluating  and  monitoring  performance,  are  likely  to  be  costly  (in 
terms  of  dollars  or  disputes,  see  Pigeon,  2006),  then  this  should  be  taken  into  account  in  the 
original  cost  estimate,  as  well  as  in  negotiating  the  optimal  contract  type.  Table  1  provides  a 
simple  illustration  of  the  spectrum  of  contract  types  that  were  awarded  to  the  top  10  defense 
contractors  over  a  six-year  period. 


Category 

Cost-Plus  (C+) 

Fixed-Price  (FP) 

Time  &  Materials 

1.  Lockheed  Martin 

50% 

47% 

2% 

2.  Boeing  Co. 

27% 

70% 

2% 

3.  Raytheon  Co. 

38% 

58% 

3% 

4.  Northrop  Grumman 

42% 

50% 

2% 

5.  General  Dynamics 

39% 

60% 

0% 

6.  SAIC 

52% 

21% 

15% 

7.  Carlyle  Group 

44% 

46% 

9% 

8.  Newport  News  Ship 

78% 

22% 

0% 

9.  TRW 

71% 

23% 

2% 

10.  Computer 

Sciences 

41% 

26% 

24% 

Table  1.  Unclassified  Details  on  the  Type  of  Contracts  Won  by  the  Top  10  Contractors  on 
Items  Outsourced  from  1998  to  2003  (Percent  of  Contracts  Awarded  that  Were  Cost-plus, 
Fixed-price  and  Time  &  Materials)  (Makison,  2004) 


18  Scope  for  opportunistic  behavior  may  lead  to  adverse  selection,  choice  of  an  (ex  ante)  inferior  option 
(or  technology),  or  moral  hazard.  Such  scope  increases  risk  that  if  a  relationship-specific  investment  is 
made,  the  other  party  will  exploit  the  terms  of  the  contract  to  “hold  them  up.”  For  example,  changes  in 
specifications  are  frequently  used  by  contractors  as  a  reason  to  raise  prices  and  profits  under  government 
contracts,  especially  when  those  investments  by  the  contractor  create  barriers  to  the  entry  of  competitors. 
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The  more  complex  and  uncertain  the  transaction,  the  less  complete  the  statement  of 
work  (performance  work  statement  (PWS)),  then  the  greater  the  cost  in  using  FP,  and  the  more 
attractive  other  contracting  options  become.19  If  the  statement  of  work  (PWS)  describing  the 
desired  product,  service  or  project  can  be  specified  precisely  (IFB),  and  there  are  no 
transaction-specific  assets  involved,  then  FP  type  contracts  have  the  benefit  of  creating  cost- 
reducing  incentives  that  reward  the  buyer  through  ex  ante  competition  between  potential 
suppliers.  In  this  case,  FP  contracting  increases  contractor  incentives  to  invest  in  cost  reduction, 
and  ex  ante  competition  can  transfer  these  cost-savings  directly  to  the  buyer. 

In  contrast,  if  the  statement  of  work  (PWS)  cannot  be  specified  precisely  (RFP)  or  if 
there  are  significant  specific  assets  involved  in  the  transaction,  then  some  surplus  will  be  eroded 
by  the  frictions  of  ex  post  re-negotiation.  This  loss  from  unproductive  bargaining  activity  is  part 
of  the  cost  of  using  a  FP  contract  in  this  case.  Initial  cost  estimates  must  take  this  into  account. 

Evidence  uncovered  by  Bajari  and  Tadelis  (1999)  reveals  that  in  cases  in  which  a 
transaction  is  easy  to  define  and  measure  (i.e.,  there  is  little  complexity),  and  only  a  few  minor 
changes  are  expected  (i.e.,  there  is  little  uncertainty),  FP-type  contracts  tend  to  dominate. 
However,  the  more  complex  the  transaction — the  more  difficult/costly  it  is  to  define  and  measure 
performance,  and  the  more  uncertain — the  more  likely  it  is  a  change  in  the  contract  will  be 
required,  the  more  severe  the  adversarial  relationships  experienced  ex  post  when  FP  contracts 
are  chosen.  In  the  latter  case,  FP-type  construction  contracts  often  ended  in  costly 
renegotiations  in  which  any  surplus  generated  was  dissipated  in  the  course  of  those 
negotiations  through  unproductive  bargaining  and  influence  activities.  Thus,  complexity  and 
uncertainty  can  force  parties  to  turn  away  from  FP  type  contracts  and  towards  C+  type 
contracts,  and  to  rely  heavily  on  reputation  and  other  enforcement  mechanisms  to  avoid  ex  post 
opportunistic  behavior  that  threatens  to  dissipate  the  surplus  generated  by  a  transaction. 

FP  (C+)-type  contracts  are  usually  prescribed  in  later  (earlier)  stages  of  product 
development  when  complexity  and  uncertainty  have  (have  not)  been  resolved,  and  the 
performance  work  statement  is  well  (not  well)  defined.  Note  that  while  these  prescribed 
contracts  focus  on  the  characteristics  of  complexity  and  uncertainty,  apparently  overlooked  are 
the  vital  roles  of  frequency  and  asset  specificity — two  key  components  of  TCE. 

In  the  case  of  frequency,  recurrent  transactions  often  justify  the  setup  costs  of 
specialized  assets  and  special  governance  requirements.  They  also  offer  the  opportunity  to 
apply  learning  curves  (cumulative  cost-quantity  relationships)  to  lower  production  costs,  and  for 
gradual  reductions  in  uncertainty  as  both  parties  learn  more  about  costs.  Recurring  transactions 
also  offer  the  possibility  for  the  accumulation  of  goodwill  and  to  build  reputations.  Strategic 
partnerships  and  relation-specific  investments  can  increase  the  benefits  to  both  parties,  but  they 


19  An  example  of  the  latter  is  Performance-based  Logistics  (PBL).  The  DoD  defines  PBL  as:  “an 
integrated  acquisition  and  sustainment  strategy  for  enhancing  weapon  system  capability  and  readiness, 
where  the  contractual  mechanisms  [...]  include  long-term  relationships  and  appropriately  structured 
incentives  with  service  providers,  both  organic  and  non-organic  [to]  support  the  end  user’s  (warfighter’s) 
objectives.”  Any  future  investments  in  PBL  could  benefit  from  the  multiple  insights  generated  by  TCE. 
The  decision  to  outsource  weapon  system  support  or  to  bundle  that  support  with  an  acquisition  and  to 
outsource  the  resulting  bundle  should  weigh  production  cost  savings  against  the  costs  and  risks 
associated  with  a  critical  source  of  supply  being  outside  the  DoD’s  control.  Those  costs  and  risks  are  part 
of  the  transaction  costs  of  outsourcing.  TCE  indicates  outsourcing  should  only  occur  if  there  are  positive 
net  savings  from  the  external  supply  relationship. 
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also  leave  them  vulnerable  to  opportunistic  behavior,  or  a  hold  up,  by  the  other  party.  These 
vulnerabilities  can  be  overcome  with  well-crafted  contracts.  However,  contracting  (a)  involves 
an  expenditure  of  resources,  and  (b)  cannot  eliminate  all  risks  associated  with  opportunistic 
behavior  from  partners  in  the  transaction.20 

The  interaction  of  opportunism  with  imperfect  and  asymmetric  information  raises  the 
possibility  of  unproductive  bargaining/influence  or  rent-seeking  activities.21  The  ultimate 
outcome — a  balance  of  productive  efforts  and  unproductive  bargaining — depends  on  the 
characteristics  of  the  transaction,  and  the  incentive  structures  that  govern  the  parties  involved, 
both  of  which  should  be  factored  into  initial  cost  estimates.22 

Where  a  transaction  requires  little  in  the  way  of  specific  assets  (no  hold  up  problem), 
and  involves  a  product  or  service  that  is  a)  well-defined  and  homogeneous  (IFB),  b)  easy  to 
measure  (limited  complexity  and  mild  information  asymmetry),  c)  routinely  used 
(recurring/frequent  purchases),  d)  not  subject  to  change  (limited  demand  uncertainty),  and  e)  is 
offered  by  competing  suppliers,  then  there  is  little  room  for  negotiation  (price  and  performance 
are  market-driven),  and  the  marginal  benefit  of  unproductive  bargaining  is  near  zero.  With  little 
room  for  bargaining  over  such  routine  and  uncomplicated  transactions,  substantial  production 
and  transaction  cost  stability  can  be  expected  in  the  defense  acquisition.  Moreover,  since 
administrative,  incentive,  and  enforcement  costs  tend  to  be  low  for  acquisitions  in  more 
contestable  (competitive)  markets,  the  marginal  cost  of  engaging  in  the  transaction  is  smaller  for 
the  DoD,  and  there  exists  an  incentive  for  the  supplier  to  invest  in  the  transaction  which 
generates  opportunities  for  cost  savings. 

In  general,  the  less  complex  and  uncertain  a  transaction,  the  easier  it  is  to  write  an 
explicit  contract  that  covers  all  relevant  contingencies.  Moreover,  the  lower  the  administrative 
and  enforcement  costs  of  that  contract,  the  lower  the  transaction  costs  associated  with  the 
contract.  These  favorable  characteristics  should  encourage  greater  productive  effort  in  the 
transaction  relationship  that,  in  turn,  should  contribute  to  lower  costs  and  better  products  that 


20  Costs  associated  with  contracting  and  the  holdup  risks  remaining  are  major  components  of 
“transactions  costs.”  The  process  of  contracting  includes  drafting  the  relevant  documents,  negotiating  a 
version  of  the  contract  that  is  signed,  taking  actions  to  enforce  that  contract,  and  renegotiating  when 
needed.  These  tasks  entail,  at  minimum,  the  services  of  skilled  people  who  develop  local  knowledge  of 
the  specific  business  relationship.  There  may  also  be  costs  associated  with  litigation,  to  include  both 
direct  (e.g.,  monetary)  and  indirect  (e.g.,  time  delay)  components.  Furthermore,  the  basic  contract  may 
well  need  considerable  administrative  and  management  attention  throughout  its  life,  even  if  full-scale 
renegotiation  is  not  undertaken.  Accomplishing  these  tasks  satisfactorily  involves  expenditure  of 
resources  and  management  attention.  Transaction  costs  (source  selection,  contract  management  and 
performance  monitoring)  can  negate  a  significant  portion  of  the  production  cost  savings  involved  with 
outsourcing. 

21  The  concept  of  unproductive  bargaining  and  rent-seeking  is  usually  attributed  to  Tullock  (1971;  1993), 
Krueger  (1974),  and  Bhagwati  (1980).  A  key  insight  of  this  literature  is  that  costly  bilateral  bargaining  by 
two  parties  for  a  bigger  share  of  the  surplus  they  jointly  create  can  dissipate  or  even  eliminate  that  surplus 
(Tullock,  1971). 

22  There  are  other  factors  as  well.  For  example,  Wolff  and  Reed  (2000)  find  significant  evidence  that,  inter 
alia ,  the  nature  of,  and  access  to,  assets  in  a  joint  venture  are  important  in  predicting  the  balance  of 
positive  sum  (productive)  and  zero  sum  (unproductive)  outcomes  for  the  participants. 
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benefit  both  parties.  Identifying  and  understanding  the  characteristics  of  the  transaction  could 
result  in  more  accurate  initial  cost  estimates. 

Alternatively,  transactions  that  involve  a  non-standard  (less  homogeneous  or  highly 
differentiated)  product  often  take  place  in  a  bilateral  monopoly  contractual  setting.  In  this  case, 
assuming  no  specific  assets  are  required,  the  results  depend  on  the  degree  of  contractual 
ambiguity  governing  the  transaction,  as  well  as  on  any  administrative  and  enforcement  costs 
involved.  However,  as  complexity,  uncertainty,  and  opportunism  due  to  specific  investments 
increase,  so  does  the  marginal  benefit  of  bargaining  or  ex  post  renegotiation.  This  results  in 
both  higher  transaction  costs  to  measure,  monitor,  and  govern  the  relationship,  as  well  as  an 
increased  risk  of  holdup. 

Productive  investment  (or  effort)  involves  two  types  of  assets:  general  and  specific.  The 
greater  the  ratio  of  specific  assets  to  total  investments  required  in  the  relationship,  the  greater 
the  risk  of  “holdup.”  Moreover,  as  the  threat  of  bilateral  dependency  increases,  the  more 
incomplete  the  contract  (and  the  lower  the  penalty  for  reneging  or  renegotiation),  the  lower  the 
marginal  cost  to  each  party  of  engaging  in  unproductive  bargaining  or  renegotiation. 

In  the  face  of  incomplete  contracting,  the  holdup  problem  poses  a  hazard  called 
“maladaptation.”  The  risk  of  maladaptation  is  captured  here  as  an  increase  in  the  return  to 
unproductive  bargaining  (for  example,  charging  high  prices  for  any  change  orders)  or  strategic 
renegotiation,  both  of  which  will  increase  costs  from  initial  estimates.  Any  time  ex  ante 
competition  among  suppliers  is  transformed  into  an  ex  post,  bilaterally  dependent  relationship, 
additional  governance  structures  may  be  required  to  induce  cooperative  adaptation.23  The 
challenge  is  to  write  a  contract  with  enough  precision  to  encourage  desired  performance,  but 
enough  flexibility  to  allow  productive  adaptation  (adjustments),  as  circumstances  require.  But  in 
the  case  of  complex  transactions  and  uncertain  outcomes,  “bounded  rationality”  precludes 
comprehensive  ex  ante  contracting  (contracts  are  inherently  incomplete)  which  raises  the 
possibility  of  gains  from  (unproductive)  ex  post  opportunistic  renegotiation  (e.g.,  the  “holdup” 
problem).  Ideally,  contracts  can  be  written  that  specify  measures  of  performance,  conflict 
resolution  procedures,  and  conditions  under  which  the  contract  can  be  modified,  as  well  as 
provisions  for  sharing  gains  from  transaction-specific  investments.24 

In  reality,  contracting  offers  an  imperfect  solution  to  opportunism.  Additional  governance 
mechanisms  to  settle  disputes  and  adapt  to  new  conditions,  ex  ante  efforts  to  screen  for 
reliability,  and  provisions  to  protect  transaction-specific  investments  may  well  ameliorate  these 
problems.  However,  governance  isn’t  costless,  and  such  agreements  often  increase  external 
transaction  costs;  and  factoring  those  considerations  into  initial  cost  estimates  may  well  improve 
their  accuracy. 


23  According  to  Williamson  &  Masten  (1999),  the  “central  problem  of  economic  organization  is  adaptation” 
(p.xi).  The  challenge  of  adaptation  is  especially  acute  when  ex  ante  competition  leads  to  ex-post 
monopoly  power.  Whenever  products,  services  or  projects  cannot  be  well  specified  in  advance  (due  to 
complexity,  uncertainty  about  future  conditions,  measurement  difficulties,  etc.),  and  they  involve 
transaction-specific  assets,  then  ex  ante  competition  (e.g.  competitive  bidding)  can  lead  to  ex-post 
monopoly/monopsony  power.  In  turn,  this  leads  to  costly  adaptation  through  bilateral  bargaining  and 
renegotiation. 

24  The  implications  of  this  paper  suggest  that  in  the  case  of  out-sourcing  a  transaction  in  which 
complexity,  uncertainty  and  asset  specificity  can  lead  to  renegotiation,  the  choice  of  governance  structure 
will  drive  productive  effort  and  unproductive  bargaining. 
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Ashley  &  Workman  (1986)  caution  that  providing  cost  incentives  in  a  contract  can  lead  to 
disagreements  and  spoiled  relationships  and  ex  post  friction  in  interpreting  the  outcomes.  In 
fact,  avoiding  these  frictions  and  reducing  the  advantages  to  renegotiation  can  be  accomplished 
by  investing  in  a  more  complete  PWS,  and  by  adopting  alternative  mechanisms  (e.g.,  reputation 
effects,  GOCO25)  to  reduce  the  return  from  opportunistic  behavior.  TCE  suggests  that  the 
degree  of  completeness  of  the  PWS  and  the  contract  is  an  optimizing  decision  by  both  parties 
that  reflects  their  trade-offs  between  an  ex  ante  investment  in  the  PWS  and  contract  design,  and 
the  potential  ex  post  cost  of  opportunistic  renegotiation. 

A  principal  insight  of  TCE  is  that  the  choice  of  optimal  governance  structure  depends  on 
the  characteristics  of  the  transaction.  Understanding  these  characteristics  can  improve  initial 
cost  estimates  by:  a)  sorting  transactions  into  categories  based  on  their  principal  characteristics 
(uncertainty,  complexity,  asset  specificity,  contestability  and  frequency),  and  b)  recognizing  the 
costs  and  consequences  of  alternative  contracts,  organizational  structures  and  mechanisms 
that  are  used  to  govern  those  transactions. 

VI.  Hypotheses:  Possible  Contributions  of  TCE 

Our  basic  hypothesis  is  that  including  TCE  considerations  (currently  an  omitted  variable) 
can  improve  cost-estimation  methodology  by  (a)  helping  to  explain  the  systematic  bias  observed 
in  initial  cost  estimates,  and  (b)  increasing  the  general  explanatory  power  of  cost  estimations. 
That  is,  we  assert  the  traditional  WBS  approach  may  overlook  some  important  variables, 
resulting  in  initial  cost  estimates  that  are  (a)  not  accurate  and  (b)  biased  toward  being 
unrealistically  low.  More  specifically,  the  TCE  perspective  suggests  the  traditional  WBS 
approach  indeed  overlooks  two  important  variables:  Coordination  Costs  and  Motivation  Costs. 
Unlike  the  production  function  approach  of  WBS,  the  TCE  approach  focuses  on  these  and  other 
key  components  of  major  weapon  system  acquisitions. 

However,  once  production  starts,  the  contractor  acquires  specialized  information  and 
assets.  Production  is  often  subject  to  economies  of  scale  and  learning  curves.  The  ability  to 
shop  around  becomes  restricted.  Even  though  there  may  be  contestability  in  the  original 
design/development  stage,  bi-lateral  monopoly  arrangements  emerge. 

The  system  program  office’s  functions/activities  related  to  monitoring,  controlling, 
information-gathering,  reporting,  decision  reviews,  enforcement,  etc.,  grow  as 
oversight/governance  increases  with  anticipated  scale  and  risk  of  investments.  Though  program 
cost  data  may  exist,  it  does  not  tell  us  the  whole  story  on  transaction  costs.  Ideally,  we  would 
want  to  find  total  program  costs  and  contract  costs.  The  difference  consists  of  transaction  costs 
(whose  main  components  are  coordination  and  motivation  costs). 

Coordination  Costs  include:  i)  Search  and  Information  Costs — to  identify  options  and 
acquire  timely,  accurate  and  relevant  information  to  evaluate  alternatives;  ii)  Bargaining  and 
Decision  Costs — to  choose  an  alternative  and  negotiate  and  write  a  contract;  and  iii)  Policing 
and  Enforcement  Costs — to  make  payments  and  measure,  monitor,  and  evaluate  performance. 

Motivation  Costs  include:  i)  Costs  to  promote  productive  effort  and  incentives  to 
encourage  investment  (better,  faster,  cheaper)  and  ii)  Costs  to  deter  unproductive  bargaining, 


25  GOCO  means  “Government  Owned,  Contractor  Operated”  production  assets. 
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and  opportunistic  behavior  (renegotiation).  Factoring  TCE  cost  considerations  into  cost¬ 
estimating  efforts  could  help  the  DoD  anticipate  cost  increases  in  four  key  areas  the  GAO 
suggests  help  explain  cost  overruns: 

a)  constantly  changing  missions  (uncertain  demand/quantity/characteristics,  bi-lateral 
monopoly,  asset  specificity,  holdup,  incomplete  contracting); 

b)  yearly  incremental  funding  vs.  multi-year  appropriations  (uncertainty,  frequency,  asset 
specificity,  holdup);  program  instability  tends  to  discourage  the  investments  that  play  a 
predominant  role  in  moving  costs  down  the  learning  curve  (McNicol,  2005,  p.26). 

c)  incentive  problems  (incomplete  contracting,  asset  specificity,  holdup);  and 

d)  insufficient  oversight  (measurement,  monitoring  costs).  (GAO,  1997) 

A.  Our  Hypotheses  in  More  Detail 

The  primary  focus  of  this  study  is  to  see  if  insights  from  TCE  can  improve  DoD  cost- 
estimation/forecasting  techniques.  Our  inquiry  can  conceptually  be  divided  into  three  questions: 

1 .  Is  cost  growth  a  problem? 

Ho:  C  5  Mean  Cost  Growth  Factor  =  Actual  Cost/Forecast  Cost  =  1 
HI:  C  >  1 

2.  Can  cost-estimating  models  which  include  TCE  insights  improve  estimated/forecast 
costs? 

Ho:  TCE  Factors  (Complexity,  Uncertainty,  Asset  Specificity,  Frequency,  etc.)  do  not 
matter — not  significant  in  explaining  cost  growth. 

Ha:  Alternatively,  Forecast  Cost  =  f  (Production  Costs;  Transaction  Costs),  where 
Production  Costs  =  g  (WBS  +  systems  integration), 

Transaction  Costs  =  Coordination  Costs  +  Motivation  Costs, 

Coordination  Costs  =  g  (Market  Contestability  Structure;  Asset  Specificity; 

Frequency;  Search  &  Information  tasks,  Contracting  & 
Enforcement  functions,  needed  Management  &  Monitoring) 

Motivation  Costs  =  h  (Complexity;  Uncertainty  [of  various  kinds],  Contract  Type  ...) 

There  is  a  fair  amount  of  evidence  about  these  hypotheses  extant.  In  a  sample  of  52 
systems,  the  GAO  found  that  RDT&E  costs  for  programs  that  started  development  with  mature 
technologies  (low  complexity/uncertainty)  increased  by  a  modest  average  of  around  5%,  while 
those  with  immature  technologies  (high  complexity/uncertainty)  experience  cost  growth  on 
average  of  almost  35%.  Similarly,  unit  costs  of  procurement  rose  by  less  than  1%  for  programs 
with  mature  technology,  whereas  programs  that  started  with  immature  technologies  saw 
increases  in  unit  costs  of  nearly  27%  over  initial  estimates.  As  predicted  by  TCE,  Complexity 
and  Uncertainty  combine  to  increase  contractual  hazards.  If  this  is  not  adequately  addressed  in 
the  contracting  phase,  then  there  is  a  greater  likelihood  of  cost  overruns. 
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B.  The  Case  for  the  Null  Hypotheses 

Broadly  stated,  the  null  hypotheses  state  that  incorporating  TCE  considerations  into  cost 
estimates  will  not  significantly  improve  cost-estimating  methodology.  In  the  course  of  our 
research,  we  found  a  surprisingly  persuasive  case  in  favor  of  that  proposition.  There  are,  in 
fact,  a  number  of  reasons  to  be  skeptical  about  our  claim  that  transactions  cost  factors  are 
important  in  determining  costs  of  major  acquisition  programs.  A  number  of  plausible  assertions 
(some  related,  some  mutually  exclusive)  underlay  such  skepticism.  While  we  find  these 
assertions  unpersuasive  on  balance,  we  cannot  dismiss  them  out  of  hand. 

1 .  Asymetric  Scope  for  Opportunistic  Behavior  in  Defense  Transactions :  It  is  more  difficult 
to  behave  opportunistically  when  involved  in  a  business  relationship  with  a  sovereign 
entity.  In  that  case,  the  government  has  more  scope  for  opportunistic  behavior,  with 
more  of  the  attendant  costs  falling  on  the  contractors.  And,  since  the  cost  estimates 
we’re  considering  refer  primarily  to  the  government’s  cost,  adding  TCE  considerations 
considers  only  minor  factors  in  those  costs. 

2.  Constancy  of  TCE  factors :  Transactions  costs  are  pretty  much  constant  for  all  major 
acquisition  programs. 

a.  If  (as  asserted  above)  the  government  has  more  scope  for  opportunistic  behavior, 
then  prospective  business  partners  will  take  that  factor  into  account  when 
considering  whether  to  compete  for  government  business — and  build  compensation 
for  those  risks  into  their  bids.  Furthermore,  such  risk  premiums  may  be  fairly 
constant  across  various  types  of  projects,  such  as  major  acquisition  programs. 

b.  Every  program  includes  an  allowance  for  managing  the  contractual  relationship 
(even  though  it’s  difficult  to  assign  program  management  overhead  completely  to 
individual  programs.)  Possibly  every  program  has  enough  of  a  management 
allowance  to  manage  the  program  reasonably  well.  Put  another  way,  the  rule-of- 
thumb  standard  for  program  management  permits  individual  program  managers  to 
deal  with  transactions  difficulties  well  into  the  range  of  diminishing  returns.  Thus, 
even  though  transactions  costs  vary  among  programs,  the  standard  program 
management  allocation  is  sufficient  to  obscure  those  differences. 

c.  Continuing  with  this  line  of  reasoning,  a  variant  of  Parkinson’s  Law  might  apply.  That 
is,  for  some  programs  the  standard  budget  for  program  management  is  too  much. 
However,  program  managers — perceiving  (probably  accurately) — that  more  program 
management  activity  has  some  marginal  return,  are  strongly  inclined  to  spend  all 
their  management  budgets. 

3.  Transactions  Cost  factors  are  already  accounted  for  in  current  cost-estimation  methods. 
Measures  of  risk  and  complexity  (such  as  technical-readiness  indicators)  indeed  indicate 
the  risk  and  complexity  of  the  project.  Also,  however,  the  presence  of  those  factors  also 
offers  increased  scope  for  opportunistic  behavior,  and  with  it  an  increase  in  transactions 
cost.  That  is,  the  transactions  cost  elements  in  major  program  costs  are  highly 
correlated  with  variables  already  considered  in  standard  cost-estimating  methods. 

4.  It  all  evens  out.  Scope  for  opportunistic  behavior  cuts  both  ways.  Scope  for  government 
opportunism  increases  (decreases)  contractors’  (government)  costs  while  contractor 
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opportunism  increases  (decreases)  government  (contractors’)  costs.  On  average,  it 
might  well  balance  out. 

5.  While  the  effects  of  TCE  considerations  are  important  and  a  very  significant  factor  in 
program  costs,  they’re  very  hard  to  find  (and  quantify).  By  definition,  opportunistic 
behavior  involves  guile  (Williamson,  1996),  and  those  who  behave  opportunistically  have 
already  taken  pains  to  conceal  their  behavior  (or  assign  other  motives  to  it). 

Therefore,  while  there  is  a  persuasive  case  of  the  importance  of  TCE  considerations  in 
defense  acquisition  processes,  it’s  not  clear  that  one  would  be  immediately  led  to  conclude  that 
incorporating  TCE  principles  into  cost-estimating  models  either  (a)  improves  their  explanatory 
power  or  (b)  ameliorates  the  low-ball  biases  we  observe  in  current  practice.  It  comes  down  to  a 
matter  of  resolving  competing  hypotheses  with  empirical  data. 

C.  Testing  Our  Hypotheses 

As  shown  in  the  sections  above,  our  research  has  uncovered  reasons  to  believe  that 
TCE  factors  are  important  in  DoD  acquisition  costs,  but  also  that  TCE  insights  may,  in  fact,  not 
improve  cost-estimation  methodology.  What’s  left  is  to  sort  out  the  competing  claims  using  the 
available  empirical  evidence. 

Our  hypothesized  set  of  relationships  is  depicted  in  Figure  3  below.  To  reject  our  null 
hypothesis,  we  need  to  do  empirical  tests,  and  have  accordingly  sought  data  on  major  DoD 
acquisition  programs.  For  Indicators  of  High  Transactions  Costs,  we  apply  the  Powell  (2002) 
stoplight  scheme,  with  special  emphasis  on  asset  specificity.  For  observable  manifestations  of 
cost  problems  and  governance  issues  during  the  program,  we  can  consult  histories  of  actual 
programs.  In  this  report,  we  use  case  studies  of  actual  Army  acquisition  programs  compiled  by 
one  of  our  team  members  (John  Dillard).  These  are  reported  in  Section  VII. 


Figure  3.  TCE  Issues  in  Acquisition  Projects  and  Hypothesized  Cost 

Manifestations 

We’ve  also  tracked  Program  Management  Office  costs  as  an  indicator  of  actual 
transactions  costs  present  within  any  given  program.  These  are  reported  in  Section  VIII. 
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VII.  TCE  Applied  to  Two  Defense  Acquisition  Case  Studies 

In  the  some  of  these  authors’  previous  work  (Dillard,  Franck  and  Melese,  2006), 
fundamentals  of  transaction  cost  economics  (TCE)  were  explained  and  insights  from  it  were 
applied  to  defense  acquisition.  TCE  asserts  that  “make-or-buy”  decisions  for  the  needs  of  a  firm 
ultimately  define  its  own  vertical  boundaries.  And  though  much  of  the  body  of  knowledge  thus 
far  consists  of  studies  from  the  private  sector,  many  public  concerns  engage  in  outsourcing  as 
well.  Defense  acquisition  is  one  such  very  large  public  enterprise  and  extends  from  R&D  and 
procurement  of  materiel  to  purchasing  services  and  sustaining  support  for  its  military  forces.  It  is 
unique  because,  for  the  most  part,  the  Department  of  Defense  outsources  much  of  what  we 
consider  to  be  “acquisition,”  commissioning  external  suppliers  to  develop  products  for  its  internal 
use,  unlike  many  private  and  public  organizations  that  conduct  internal  product  development  for 
themselves  and  others  (or  external  products/services  for  others).  TCE  acknowledges  that 
outsourcing  relationships  can  vary  widely  in  their  transactional  characteristics  and  can  involve 
extra  transaction  costs  such  as  measurement,  monitoring,  and  negotiation  costs  that  could 
negate  the  initially  perceived  advantages  of  the  buy-versus-make  decision. 

TCE  is  descriptive  of  economic  behavior  and  recognizes  the  issue  of  motivation.  It 
assumes  that  economic  actors,  such  as  government  acquiring  “principals”  and  defense  industry 
supplying  “agents,”  are  motivated  to  forecast  potential  outcomes  and  factor  these  into  contracts. 
The  DoD’s  general  motivation  (among,  no  doubt,  other  agendas)  in  its  acquisition  transactions 
is  the  seeking  of  better  products,  delivered  quickly,  and  with  fewest  resources  (i.e., 
performance,  schedule,  and  cost).  But  such  outsourcing  is  complicated  by  other  phenomena. 
TCE  characteristics  of  complexity  (such  as  technology)  and  uncertainty  (such  as  duration  of 
work)  apply  clearly  to  many  Defense  acquisition  transactions  and  lead  to  “incomplete 
contracting,”  termed  as  such  from  imperfect  information  and  our  inability  to  predict  the  future. 
These  endanger  a  firm’s  ability  to  protect  its  own  interest  throughout  the  transaction,  but 
hopefully  are  mitigated  via  governance  mechanisms  that  positively  influence  a  supplier’s 
motivation  to  comply  with  the  terms  of  an  incomplete  contract.  These  might  be  via  contract 
incentives,  enforcement  mechanisms,  monitoring  methods,  etc. 

Another  often  encountered  but  heretofore  largely  unrecognized  TCE  characteristic  (at 
least  in  Defense  acquisition  literature)  is  that  of  asset  specificity — a  situation  in  which  a  supplier 
“locks-in”  the  government  by  making  investments  in  productive  assets  that  are  specific  to  the 
transaction  and  have  little  value  elsewhere.  Asset  specificity  can  be  related  to  physical  capital 
equipment,  human  skills,  facility  location,  and  even  brand  equity  (reputation).  While  needed  to 
accomplish  the  work  at  hand,  the  result  of  these  investments  form  barriers  to  competitive  entry 
and  can  also  result  in  a  “holdup.”  A  holdup  is  a  problem  that  occurs  when,  for  example,  the 
agent  in  a  contract  has  a  specific  asset  with  concerns  that,  after  its  investment,  may  have  to 
accept  worse  terms  than  anticipated.  Conversely,  the  principal  has  become  increasingly 
dependent  on  the  agent  during  his  investment  in  the  specific  assets.  It  has  become  fairly  typical 
in  the  DoD  acquisition  experience:  with  development  contracts  that  are  necessarily  incomplete, 
sometimes  won  by  low  bids,  and  changes  in  contract  scope  later  arise.  The  contractor  can 
anticipate  higher  returns  from  “holding  up”  the  government  for  such  contract  modifications.  And 
while  the  government  might  have  the  sovereign  right  to  terminate  the  contract  for  convenience, 
it  will  still  be  left  with  the  demand  for  the  product  or  service  that  was  sought.  Such  has  been  the 
case  in  several  programs  observed  by  these  authors,  along  with  excursions  into  contract  type 
and  structure  as  options  for  obtaining  desired  outcomes. 
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Acquisition  Case  Examples 


One  of  the  authors  was  fortunate  to  have  served  as  the  Assistant  Project  Manager  for 
Research  &  Development  in  two  separate  major  defense  acquisition  programs  that  were 
offspring  of  Defense  Advanced  Research  Projects  Agency  (DARPA)  initiatives.  These  programs 
are  described  below  to  illustrate  TCE  challenges  and  uses  of  governance  mechanisms. 

The  Javelin  Project 

The  Advanced  Anti-Armor  Weapon  System — Medium  (AAWS-M),  later  to  become  the 
Javelin,  began  in  1982  as  the  DARPA  program  “Tank  Breaker.”26  This  was  a  one-year 
technology  demonstration  to  explore  various  missile-guidance  solutions  for  a  medium  range 
(i.e. ,  1-2000  meters)  man-portable,  anti-tank  weapon.  It  was  spawned  as  a  result  of  deficiencies 
that  were  immediately  apparent  in  the  recently  fielded  DRAGON  weapon  system,  which  had 
replaced  the  M67  90mm  recoilless  rifle  in  the  late  1970s.  The  DRAGON  was  a  wire-guided, 
line-of-sight  missile  that  was  developed  in  response  to  the  1960s  appearance  of  the  Soviet  AT-3 
SAGGER,  a  manpack  missile  carried  in  a  fiberglass  "suitcase."  In  1978,  a  Mission  Need 
Statement  highlighted  deficiencies  of  the  DRAGON,  such  as  its  poor  reliability,  limited 
range/lethality,  and  the  difficulty  for  gunners  to  aim  and  track  targets.  The  envisioned 
replacement  was  to  satisfy  a  substantial  increase  in  requirements — namely:  range,  lethality, 
reduced  weight,  and  the  ability  to  launch  from  enclosures  (such  as  buildings  or  field-fortified 
bunkers).  Several  years  were  spent  finalizing  these  requirements  until  the  joint  Army  and  Marine 
Corps  operational  requirements  document  was  formally  approved  in  1986-88.  A  competitive  fly- 
off  program,  which  would  now  be  called  the  “Technology  Development  phase,”  was  conducted 
in  1987-1989  to  select  from  three  teams  of  contractors  and  critical  technologies:  a  laser-beam 
rider  led  by  Ford  Aerospace,  a  fiber-optic  guidance  effort  led  by  Hughes,  and  a  forward-looking 
infrared  (FUR)  thermal  imaging  sensor  effort  from  Texas  Instruments  and  Martin-Marietta. 
Clearly,  the  down-selection  to  one  team  was  of  great  incentive  to  the  participants,  for  it  meant 
the  likely  follow-on  award  of  the  advanced  development  phase  as  well  as  production 
opportunities.  Cost  Plus  Fixed  Fee  (CPFF)  contracts  were  used  with  each  of  the  three  teams. 
Such  is  typical  for  “proof-of-principle”  type  R&D  efforts  in  which  knowledge  work  is  risky  for  the 
contractor,  and  there  are  few  discrete  parameters  to  incentivize.  All  three  teams  were 
successful  in  flying  missiles  to  their  targets,  but  the  only  technology  that  enabled  a  true  fire-and- 
forget  capability  (which  was  not  a  specified  requirement)  was  the  FUR  approach.  Though  this 
approach  was  recognized  to  be  the  most  technically  immature  and  risky,  the  desire  for  fire-and- 
forget  survivability  resulted  in  the  FUR  team  being  awarded  a  contract  for  a  three-year 
advanced-development  phase. 

In  June  of  1989,  a  full-scale  development  (now  called  System  Development  and 
Demonstration)  contract  was  awarded  for  the  Javelin  program.  The  program  was  structured  to 
encourage  competition  and  give  incentives  for  the  accomplishment  of  all  of  the  system 
objectives  of  the  Department  of  Defense.  First  of  all,  the  contract  would  be  awarded  to  a  team  of 
two  partnering  firms  that  would  combine  their  efforts  for  development  of  the  system  and  split 
apart  to  compete  during  the  production  phase.  The  initial  low-rate  production  was  envisioned  to 
be  a  competitive  split  awarded  annually  with  a  fifty/fifty  or  sixty/forty  split  between  partners,  via 
the  use  of  fixed-price  contracts.  Such  contracts  are  also  typical  for  production  efforts  in  which 
the  design  (uncertainty  and  complexity)  is  presumed  to  have  evolved  to  a  point  of  stability.  At 


26  DTIC  reports  summarizing  this  DARPA  effort  are  available  at 

http://stinet.dtic.  mil/oai/oai?&verb=qetRecord&metadataPrefix=html&identifier-ADA1 22234. 
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the  macro  level,  the  office  of  the  Secretary  of  Defense  viewed  the  program  as  acceptable  with 
regard  to  risk  because  of  its  27-month  technology  development  phase  and  subsequent  36- 
month  plan  for  full-scale  development. 

During  the  technology-development  phase,  all  three  contractor  teams  had  scored  over 
62%  hits  with  at  least  ten  missile  shots  each  in  a  variety  of  environments  and  operational 
settings.  The  full-scale  development  contract  request  for  proposal  was  written  for  a  cost-plus- 
incentive-fee  type  of  contract,  but  the  winning  proposal  was  presented  with  a  “no-fee”  bid.  This 
surprised  the  Government,  which  nevertheless  awarded  a  cost-plus-incentive-fee  contract, 
giving  incentives  for  key  performance  parameters  such  as  weight  and  warhead  performance 
that  it  considered  technically  risky.  The  total  value  of  the  contract  was  $169.7  million,  the 
amount  bid  by  the  winning  team  of  Texas  Instruments  and  Martin-Marietta.  Meanwhile,  the 
Government  privately  conducted  its  own  should-cost  estimate  and  budgeted  $263  million  for  the 
36-month  long  development  effort.  In  addition,  the  Government  ran  its  own  alternate  warhead 
technology  development  program  with  CMS  acting  as  the  contractor. 

The  two-partner  Joint  Venture  in  full-scale  development  was  also  free  to  maximize 
competition  at  the  subcontractor  level.  In  their  make-versus-buy  decision,  Texas  Instruments 
elected  to  make  the  focal  plane  array  for  both  of  its  uses  in  the  command  launch  unit  and  in  the 
missile.  They  had  made  these  devises  for  other  programs  but  not  in  these  two  distinct 
configurations.  Their  physical-specific  assets  were  located  at  the  manufacturing  facility  in 
Lewisville,  Texas,  largely  for  the  promise  of  the  Javelin  system’s  later  production.  They  had 
expanded  that  facility  for  the  Javelin  program  in  anticipation  of  producing  some  58,000  to 
70,000  missiles  and  5,000  command  launch  units  (the  thermal  imaging  sighting  device). 

Each  missile  focal  plane  array  was  forecasted  to  be  approximately  15%  of  the  missile's 
cost  (or  about  $12,000  for  each  of  the  focal  plane  arrays)  based  upon  a  $90,000  “cost  per  kill" 
program  objective.  Focal  plane  array  technology  was  still  immature  and  would  be  gauged  today 
at  approximately  technology  readiness  level  five  (on  a  1-9  scale),  despite  its  successful 
technology  development  phase  results.  It  was  always  recognized  as  technologically  risky,  so  the 
Government  partially  funded  other  companies  that  could  produce  these  devices.  In  1991 ,  the 
only  five  known  FPA  makers  in  the  world  were  Rockwell  International,  Loral,  Santa  Barbara 
Research  Corporation,  Sofradir  (a  French  firm),  and  Texas  Instruments. 

As  an  additional  gauge  of  technological  maturity,  a  baseline  test  was  mandated  at  the 
second  milestone  upon  the  decision  to  launch  the  Javelin  program  into  full-scale  development. 
That  test  would  pit  the  immature  focal  plane  array  technology  against  existing  TOW  and 
DRAGON  (legacy  systems)  missile  optics.  Results  of  this  test  showed  the  Javelin's  immature 
focal  plane  arrays  to  be  substantially  better  in  performance  than  the  DRAGON  and  almost  as 
good  or  as  good  as  the  larger  TOW  anti-tank  missile  system. 

Approximately  18  months  after  the  full-scale  development  phase  contract  award,  the 
Javelin  project  manager  called  for  Defense  Acquisition  Board,  forecasting  a  Nunn-McCurdy 
breach  of  cost  and  scheduling  thresholds  in  this  ACAT  1-D  program.  Several  reasons  were 
cited,  not  the  least  of  which  was  that  the  focal  plane  array  production  yield  was  not  as 
predicted — and  all  of  the  devices  were  below  specification.  The  cost  growth  was  found 
throughout  the  various  elements  of  the  project,  as  illustrated  in  Table  2  below. 

Over  the  next  year,  the  program  was  given  a  new  baseline  with  many  different  revised 
program  estimates  climbing  from  36  months  duration  and  $298  million  in  cost,  to  48  months 
duration  and  $372  million  in  cost,  and  then  to  54  months  and  $420  million  for  the  total  cost  and 
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duration  of  this  phase.  In  addition  to  the  rebaselining,  the  Government  renegotiated  its  cost- 
plus-incentive-fee  contract  with  the  joint  venture  and  established  cost  sharing  for  any 
expenditure  above  $372  million,  all  the  way  to  the  contract  ceiling  of  $420  million. 

This  was  a  very  unique  move,  to  re-establish  governance  mechanisms  in  the  middle  of  a 
transaction.  But  the  contractor’s  “failure  to  perform,”  although  in  an  uncertain  R&D  environment, 
justified  the  new  relationship.  At  30  months  after  contract  award,  the  program  was  finally 
formally  re-baselined  to  be  54  months  and  $443  million,  with  50/50  JV  and  Government  cost¬ 
sharing  ratio  above  that  amount. 

In  October  of  1991  at  a  meeting  in  the  Lewisville,  Texas,  focal  plane  array  manufacturing 
facility,  the  “divorce”  finally  came  for  Tl  as  prime  vendor  of  the  missile  focal  plane  array.  Texas 
Instruments  had  failed  to  achieve  specification  for  the  item  by  the  target  date,  but  they  still  had  a 
stake  in  the  program  as  one  of  the  principal  players  on  the  joint  venture  team. 

About  this  same  time,  the  Government  discovered  a  focal  plane  array  “holding  account” 
that  had  been  proposed  and  was  awarded  in  scope  of  the  contract.  Essentially,  this  holding 
account  accrued  charges  to  the  Government  for  every  non-specification  compliant  focal  plane 
array  that  Tl  had  produced.  The  cumulative  total  was  to  be  billed  all  at  once,  when  they 
achieved  the  first  specification-compliant  focal  plane  array. 
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CONTRACT  COST  GROWTH 
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ORIGINAL 

BUDGET 

CURRENT 

LRE 

DELTA 

1.  ROUND  A  MISSII.F. 

1 1.733 

20.041 

(4315) 

2.  PROPULSION 

2,327 

4386 

(6J059) 

3.  WARHEAD/CONTACT  FUZE 

9.720 

13,661 

(3.941) 

4.  ELECTRONIC  SAFE  AND  FIRE 

1.459 

6.926 

(5.467) 

5.  GUIDANCE  SECTION 

16.092 

35,662 

(19.570) 

6.  SEEKER 

INCLUDES  FOCAL  PLANE  ARRAY 

24.227 

(14000) 

47.905 

13.7.000) 

(23.678) 

(19,000) 

7.  CONTROL  ACTUATION  SYSTEM 

2.046 

5.180 

(3.134) 

8.  TELEMETRY 

2.405 

4.085 

(1,680) 

9.  LAUNCH  TUBE  ASSEMBLY 

vm 

6383 

(4347) 

10.  BATTERY  COOLANT  UNIT 

1.155 

4.849 

(3>9<) 

II.  ROUND  SHIPPING  CONTAINER 

107 

2.420 

0313) 

11  COMMAND  LAUNCH  UNIT 

INCLUDES  DETECTOR  DEWAR  COOLER 

18.976 

I2.60OJ 

53JH2 

(15.100) 

(34.836) 

(12.5001 

13.  TRAINING  DEVICES 

7.155 

9.636 

(1.881) 

14.  TEST  &  EVALUATION 

11.260 

12310 

(1.250) 

15.  SYSTEM  ENGINEER INO 

9^62 

21.606 

(12.244) 

16.  PROJECT  MANAGEMENT 

35.298 

64396 

(29398) 

17.  INTEGRATED  LOGISTICS  SUPPORT 

3.949 

6,010 

0.121) 

18.  DATA 

4.026 

4,505 

(479) 

19.  JOINT  VENTURE  GAA 

11.591 

20.247 

(8.656) 

ALL  OTHER 

4.559 

13.465 

(4.906) 

TOTAL 

180.043* 

361152 

(182 ,060) 

‘3IM7M  ♦  COfTUACT  MOOS 


Table  2.  Early  1991  Summary  of  Javelin  Program  Component  Cost  Growth 

Since  the  contract  was  cost-reimbursable  in  form,  the  Government  was  likely  to  have  to 
pay  for  all  costs  of  development  regardless  of  spec  compliance,  but  the  way  the  billing 
transpired  took  the  Government  by  surprise.  Fortunately,  dual  sourcing  of  critical  components  is 
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a  fundamental  element  in  acquisition  strategy  development  and  risk  mitigation.  And  the 
selection  of  Hughes’s  Santa  Barbara  Research  Corporation  as  a  sub-contractor  to  the  other 
Joint  Venture  partner,  Martin -Marietta,  saved  the  Javelin  program.  That  small  company  had 
discovered  a  process  to  produce  near-perfect  focal  plane  arrays  repeatably,  and  they  were 
accelerated  to  provide  these  for  the  program.  The  investment  was,  in  hindsight,  a  “real  option” 
that  was  exercised  mid-project  with  great  return  on  investment. 

Within  that  year,  the  program  was  restructured,  given  the  new  baseline,  and  finished 
largely  within  its  new  parameters.  The  additional  18  months  added  to  the  36-month  phase 
helped  resolve  the  uncertainties  and  complexities  of  system  development  without  additional 
schedule  slippage.  Later,  production  quantities  were  slashed  in  half  as  the  Defense  Department 
drew  down  its  forces  from  1991-2000,  and  the  acquisition  strategy  to  split  apart  the  joint  venture 
and  compete  them  in  production  was  not  fulfilled.  Benefits  of  a  split  production  no  longer  able  to 
be  realized,  the  JV  remained  intact  as  the  producing  entity.  Today  Javelin  is  seen  as  a 
successful  weapon  system,  is  being  used  in  Iraq  and  Afghanistan  and  has  been  through  several 
full-rate  production  contract  periods. 

The  Army  Tactical  Missile  System  Project 

The  Army  Tactical  Missile  System  also  started  out  as  a  DARPA  program,  called  Assault 
Breaker,  in  the  1970s.  The  system's  prime  contractor  for  the  vehicle,  the  Multiple  Launch  Rocket 
System,  or  MLRS,  was  LTV  Corporation.  With  significant  asset  specificity  from  their  platform 
efforts  (and  physical  &  knowledge  capital),  they  bid  upon  and  won  two  separate  contracts  for  the 
Army  Tactical  Missile  System.  These  were  both  firm-fixed-price:  one  for  the  invention  of  the 
missile  and  one  for  its  integration  into  the  platform.  The  program  was  a  48-month  full-scale 
development  effort  that  was  begun  in  1986  and  ended  in  1990.  In  stark  contrast  to  the  Javelin 
program,  fixed-price  contracts  were  deemed  appropriate  to  this  transaction  because  of  the 
technology  maturity  in  the  ATACMS  project,  relative  to  most  others,  where  CPIF  contracts  are 
common — as  depicted  in  Table  3. 
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Key  Program  Characteristics  -  First  Increment  of  Capability 

Program  Aspects 

ATACMS 

JAVELIN 

DARPA  Predecessor 

Assault  Breaker  1977-82 

Tank  Breaker  1981-82 

Ultimate  Capability 

“Deep  Attack” 

"Fire  &  Forget” 

Critical  Technoloaies  &  Readiness  Levels: 

Munition 

9  -  Lance  M74  Bomblet 

5-  Tandem  Shaped  Charges 

Propulsion 

9  -  Solid  Rocket  Motor 

5-  Two-Stage  Solid  Rocket  Motor 

Flight  Control 

9  -  Fin  surfaces 

6  -  Fins  +  Thrust  Vector  Control  Vanes 

Guidance  and  Control 

9  -  Inertial 

4  -  Tracker  Software  Algorithm 

Safe/Arm  Fusing 

7  -  Mechanical 

4  -  Electronic 

Software  Function  (Target  Acquisition.  Fire  Control,  etc.) 

6  -  Various 

6  -  Various 

Sensor 

N/A 

5  -  Focal  Plane  Array 

Capability  Leap  Area 

Range 

Range.  Lethality.  Survivability 

Cost  of  development 

~$700M 

~$700M 

Contract  Type 

Fixed  Price 

Cost  Reimbursable 

Tech  Development  Phase 

0  Months 

27  Months 

Advanced  Development  Phase  -  Planned 

48  Months 

36  Months 

Advanced  Development  Phase  -  Actual 

51  Months 

54  Months 

Total  Time  in  Development 

51  Months 

81  Months 

Advanced  Development  Phase  Contract  Cost  Growth  0% 

>150% 

Table  3.  Comparison  of  Programs  Using  Different  Contract  Types  and 
Technology  Readiness  Levels 

Also,  this  program  was  structured  to  have  production  options,  should  the  full-scale 
development  phase  be  successful.  Thus,  there  were  incentives  for  “performance”  on  both  the 
Government  and  the  contractor  side:  LTV  was  given  an  incentive  to  succeed  in  the  development 
of  the  missile  and  its  integration  in  order  to  win  the  follow-on  low-  and  full-rate  production 
contracts.  And  likewise,  the  Government  had  an  “incentive”  to  conduct  reviews  and  make 
decisions  to  award  production  by  November  of  1990.  This  was  to  preserve  production-pricing 
options  that  the  contractor  had  been  asked  to  provide  in  his  proposal  for  advanced 
development.  The  price  reduction  from  these  options,  if  exercised  before  expiration,  was  on  the 
order  of  40%.  As  it  turned  out,  the  production  options  were  exercised  within  a  week  of  their 
expiration,  saving  the  Government  several  hundred  thousand  dollars  per  missile  on  the  eve  of 
the  first  Persian  Gulf  War. 

These  cases  may  only  serve  to  illustrate  several  recognized  economic  behaviors  within 
the  context  of  large  acquisition  transactions.  As  an  adjunct  to  our  earlier  work  (Dillard,  Franck  & 
Melese,  2006),  they  show  how  complexity  and  uncertainty  of  development  projects  relate  to 
contract  type  and  governance  mechanisms.  Importantly,  they  also  convey  the  need  to  constrain 
scope  in  order  to  control  cost  and  schedule.  In  and  of  themselves,  they  cannot  necessarily 
prove  cause  and  effect,  but  rather  reveal  the  importance  of  understanding  key  characteristics  of 
a  transaction  early  in  the  cost-estimation  process  in  order  to  capture  relevant  transaction  cost 
elements  that  can  impact  the  ultimate  price  paid  for  major  weapon  systems. 

TCE  Assessment  of  the  Case  Studies 

As  indicated  in  Section  VI  above,  we  looked  for  observable  effects  of  TCE  factors  in 
actual  DoD  acquisition  programs.  This  section  analyzes  the  two  cases  (Javelin  and  ATACMS) 
described  above  in  terms  of:  (1)  ex  ante  indicators  of  transactions  costs,  (2)  manifestations  of 
motivation  and  coordination  problems  in  the  course  of  the  program,  and  (3)  cost  overruns 
observable  at  program  completion. 
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The  ex  ante  indicators  come  from  a  scheme  proposed  by  Powell  (2002)  and  augmented 
by  Franck  (2004).  The  indicators  of  transaction  costs  considered  here  are  asset  specificity, 
complexity,  length  of  the  relationship  (or  transaction),  time  sensitivity  of  performance,  and 
operational  significance  of  performance. 


TCE  Indicator 

Assessment 

Comments 

Asset  Specificity 

YELLOW 

Tl’s  insourcing  of  FPA  production. 

Mitigated  by  planned  dual-source 
production,  and  steps  to  diversify  FPA 
sources. 

Complexity 

RED 

Fire-and-forget  feature  added 
significantly  to  complexity. 

Length  of 

Relationship 

YELLOW 

Technical  immaturity  necessitated  a 
lengthy  development  program. 

Time  Sensitivity 

YELLOW 

Green  after  end  of  Cold  War. 

Operational 

Significance 

YELLOW 

Green  after  end  of  Cold  War 

Table  4.  Ex  Ante  Assessment  of  Javelin  Development  Program 


TCE  Indicator 

Assessment 

Comments 

Asset  Specificity 

RED 

Pre-existing  condition.  Contractor’s 
previous  experience  with  launch 
vehicle.  Production  option  proved  a 
hedge  for  the  contractor. 

Complexity 

GREEN 

Technology  generally  mature 

Length  of 

GREEN 

Advanced  Development  Phase 

Relationship 

only. 

Time  Sensitivity 

YELLOW 

Green  after  end  of  Cold  War 

Operational 

Significance 

YELLOW 

Green  after  end  of  Cold  War 

Table  5.  Ex  Ante  Assessment  of  ATACMS  Development  Program 

(Powell  (2002),  Franck  (2004)  and  authors’  assessments) 


The  relative  immaturity  of  Javelin  technologies  necessitated  a  long  and  complex 
transaction.  Worth  noting  is  that  a  government  decision,  in  favor  of  the  fire-and-forget  feature, 
increased  these  difficulties.27  By  contrast,  the  ATACMS  development  program  was  based 
largely  on  proven  technologies.  For  both  development  programs,  Time  Sensitivity  and 
Operational  Significance  decreased  considerably  with  the  end  of  the  Cold  War.  Both  ATACMS 
and  Javelin  were  designed  to  counter  massive  Warsaw  Pact  mechanized  offensives,  at  the 


27  We  have  no  basis  to  question  (from  an  ex  ante  perspective)  the  Army’s  decision  to  opt  for  a  less 
mature  technology  in  pursuit  of  significantly  higher  performance.  However,  we  believe  consideration  of 
the  increase  in  scope  for  opportunistic  behavior  in  the  program  should  be  considered  when  making  such 
decisions. 
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operational  and  tactical  levels  respectively.  With  the  end  of  the  Cold  War,  both  priority 
(operational  significance)  and  urgency  (time  sensitivity)  lessened  accordingly.  Arguably,  this 
development  significantly  strengthened  the  government’s  position  in  the  negotiations  associated 
with  governance  of  both  programs. 

Observable  manifestations  of  motivation  and  coordination  difficulties  during  the  course  of 
the  project  are  gleaned  from  the  narratives  above.  For  Javelin,  there  were  a  number  of 
difficulties  apparent  throughout  the  problem,  predictable  from  a  TCE  perspective.  These 
included  the  following. 

■  The  winning  bid  clearly  seems  to  have  been  a  buy-in.  The  contracting  team’s  initial 
estimate  was  significantly  below  the  government’s  estimate  of  most  likely  program  cost. 
From  a  TCE  perspective,  this  can  be  interpreted  as  setting  the  stage  for  opportunistic 
behavior  on  behalf  of  the  contractors. 

■  One  member  (Texas  Instruments)  of  the  development  team  (the  Joint  Venture)  chose  to 
make  the  Focal  Plane  Arrays  (FPAs)  within  firm  boundaries.  The  FPAs  were  arguably 
the  critical,  and  pacing,  element  of  the  Javelin  development  program.  Whether  this 
indicates  opportunistic  behavior  by  Tl,  or  simply  a  miscalculation,  is  difficult  to  assess. 
Worth  noting,  however,  is  that  the  Army  took  steps  toward  developing  an  alternate 
source.  A  TCE  perspective  would  indicate  the  government  perceived  some  scope  for 
opportunistic  behavior;  a  risk-management  perspective  would  interpret  this  as  a  risk 
reduction  measure. 

■  There  was  steady,  and  cumulatively  very  significant,  cost  growth  throughout  the 
program.  As  a  result,  the  Program  Manager  reported  breaches  of  criteria  for  both  cost 
and  schedule — relatively  early  during  the  program. 

■  Other  cost-growth  problems  appeared.  For  example,  the  bill  for  FPA’s  not  meeting 
specifications  was  presented  to  the  government  without  previous  notice.  This  turned  out 
to  be  an  unpleasant  surprise.  Whether  this  was  simply  a  communications  difficulty  or 
opportunistic  behavior  on  the  part  of  the  Joint  Venture  is  not  clear. 

■  Consequently,  governance  of  the  transaction  became  significantly  more  complicated. 
Major  changes  to  program  structure  were  negotiated  as  part  of  a  “re-baselining.” 
Additionally,  the  contract’s  incentive  structure  was  changed —  from  a  cost-plus  to  a  risk¬ 
sharing  form. 

Overall,  there  was  significant  cost  growth  (over  150%)  in  the  Javelin  development 
program. 

For  ATACMS,  the  only  major  indicator  of  transactions  cost-related  difficulties  was  asset 
specificity.  LTV’s  previous  experience  with  the  launch  vehicle  (the  MLRS)  gave  it  a  substantial 
advantage  over  alternate  vendors — making  this  an  excellent  example  of  what  can  happen  with 
asset  specificity.  However,  the  program  was  executed  on  schedule  and  within  the  original  cost 
estimate.  It’s  possible  that  LTV  had  no  incentive  to  low-ball  its  original  estimate  with  its  already 
significant  advantages  over  potential  competitors  based  on  asset  specificity. 

There  was  arguably  evidence  of  opportunistic  behavior  on  the  part  of  the  government. 

■  A  program  based  on  a  fixed-price  contract  was  extended  by  three  months  (from  48  to 
51)  simply  because  of  governmental  convenience — without  compensation  to  the 
contractor. 
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■  Also,  the  government  was  slow  to  exercise  its  option  to  order  the  start  of  production,  and 
would  likely  have  delayed  its  decision  further  were  it  not  for  the  imminent  expiration  of  a 
favorable  price  option  in  the  contract.  It’s  reasonable  to  suppose  that  this  part  of  the 
contract  served  to  head  off  opportunistic  behavior  on  the  part  of  the  government. 

Some  observations  are  useful  at  this  point.  First,  the  indicators  of  high  transactions 
costs  should  be  considered  in  toto.  As  we’ve  stated  above,  there  are  excellent  reasons  to 
believe  that  asset  specificity  is  particularly  important  in  creating  conditions  for  opportunistic 
behavior  in  a  transaction.  However,  in  the  case  of  ATACMS,  there  appears  to  have  been  no 
bad  behavior  that  added  significantly  to  program  outcomes  (cost,  schedule  or  performance), 
despite  the  RED  assessment  for  asset  specificity. 

Second,  opportunistic  behavior  in  transactions  is  a  question  of  motives,  and  it’s  difficult 
to  sort  out  motives  in  case  narratives.  For  example,  was  the  contractors’  original  estimate  for 
Javelin  development  a  buy-in  or  a  manifestation  of  the  Winner’s  Curse?  Was  the  unexpected 
presentation  of  the  bill  for  substandard  Javelin  FPAs  a  matter  of  opportunistic  behavior  (self- 
interest  seeking  with  guile)  or  simply  an  administrative  failure  to  communicate?  Without 
considerably  more  information  that’s  available  in  the  public  record,  it’s  difficult  to  sort  out  such 
matters.28 

However,  finally,  there  is  good  evidence  that  TCE  perspectives  offer  significant 
explanatory  power  in  analyzing  program  outcomes — including  cost.  Based  on  this  sample  of 
two,  however,  it’s  explanation  of  the  qualitative  sort.  Our  next  section  (VIII)  explores 
quantitative  evidence  of  transactions  costs  at  a  macro  level. 

VIII.  Program  Office  Costs  and  TCE 

In  order  to  test  our  hypothesis  that  the  traditional  WBS  approach  may  overlook  some 
important  variables  resulting  in  unrealistically  low  initial  cost  estimates,  we  would  have  to 
compare  cost  estimates  for  systems  that  included  significant  transaction  costs  with  those  of 
systems  that  did  not  include  significant  transaction  costs.  The  first  problem,  then,  was  to  find  a 
way  to  measure  transaction  costs  in  acquisition  programs.  We  proposed  using  Program 
Management  Office  (PMO)  costs  as  a  proxy  measure  of  the  amount  of  transaction  costs  present 
in  an  acquisition  program. 

We  started  by  examining  information  from  the  Consolidated  Acquisition  Reporting 
System  (CARS)  to  find  evidence  of  transaction  costs.  The  information  is  contained  in  the 
Defense  Acquisition  University  (DAU)  Business  Information  Laboratory  (BIL)  database  managed 
by  OUSD(AT&L)  Acquisition  Resources  and  Analysis.  It  includes  information  on  contract 
performance  and  program  cost  from  a  variety  of  reports,  such  as  Selected  Acquisition  Reports 
(SAR)  and  Defense  Acquisition  Executive  Summaries  (DAES),  as  well  as  other  reports. 
Unfortunately,  these  reports  do  not  contain  the  level  of  detail  necessary  to  identify  transaction 
costs.  Specifically,  there  was  no  information  on  the  amount  of  resources  estimated  or  used  for 
the  PMO. 

Instead,  we  looked  at  the  Budget  Item  Justification  sheets  in  the  OSD  budget 
(http://www.defenselink.mil/comptroller/budgetindex.html).  While  there  is  some  information  on 
PMO  costs  in  these  documents,  it  is  reported  inconsistently  or  not  at  all  (depending  on  the 


28 


The  obvious  incentives  to  conceal  the  motives  underlying  self-seeking  behavior  add  to  this  difficulty. 
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program  and  year).  For  example,  the  Marine  Corps  Advanced  Amphibious  Assault  Vehicle 
(AAAV)29  reported  Program  Office  costs  in  exhibit  R-3  under  “Support  and  Management 
Organizations”  for  FY97  and  FY98,  but  discontinued  reporting  that  line-item  in  subsequent 
years.  Note  also  that  what  is  included  in  PMO  costs  is  not  a  complete  picture  of  the  resources 
used,  since  military  salaries  are  excluded  and  civilian  salaries  may  or  may  not  be  included 
depending  on  how  they  are  funded. 

The  AAAV  reported  costs  for  the  more  general  category  “Support  and  Management”  for 
FY97  -  FY06,  but  the  line-items  included  in  this  category  varied  from  year  to  year.  If  we  expand 
our  proxy  measure  to  the  “Support  and  Management”  category,  we  are  including,  in  addition  to 
Program  Office  costs,  other  support  contracts,  miscellaneous  contracts  and  government  labs, 
as  well  as  modeling  and  simulation.  As  the  program  developed,  this  category  grew  to  include 
integrated  logistics  support,  training  devices  and  simulators,  tech  data  and  publications 
development  and  support  equipment  development.  Clearly,  this  category  includes  costs  that 
should  not  be  considered  transaction  costs,  such  as  training  devices  and  simulators  and  tech 
data  and  publications.  More  importantly,  what  is  and  is  not  included  in  the  category  varies  over 
time,  making  the  identification  of  transaction  costs  difficult  on  a  case-by-case  basis  and  nearly 
impossible  on  a  large  scale. 

A  more  significant  problem  we  encountered  is  that  the  information  reported  in  CARS 
does  not  necessarily  track  to  the  information  reported  for  the  same  program  in  the  OSD  budget. 
This  problem  was  confirmed  by  OUSD(AT&L)  Acquisition  Resources  and  Analysis  and  is  an 
issue  they  have  been  working  on  for  over  3  years  and  have  documented  in  a  Comparison 
Report  that  identifies  potential  FY07  funding  disconnects  between  the  OUSD(Comptroller) 
Budget  Justification  Materials  and  the  OUSD(AT&L)  Draft  Selected  Acquisition  Reports. 

Contributing  to  the  difficulty  of  identifying  program  transaction  costs  is  the  fact  that 
program  managers  only  report  information  on  a  program's  major  contracts  for  RDT&E, 
procurement,  military  construction,  and  acquisition-related  operation  and  maintenance. 
According  to  the  CARS  Users’  Guide,  SAR  Section  15  (Contract  Information)  only  includes  the 
six  largest,  currently  active  contracts  (excludes  subcontracts)  that  exceed  $40  million  in  then- 
year  dollars.  For  a  given  reporting  quarter,  these  are  generally  the  same  contracts  reporting  in 
Section  6  (Program  Background  Data)  of  the  DAES.  If  a  previously  reported  contract  is  over 
90%  complete,  it  will  no  longer  be  reported.  So,  tracking  Budget  at  Completion  (BAC)  and 
Estimate  at  Completion  (EAC)  at  the  program  level  involves  moving  targets  as  the  individual 
contracts  are  completed  and  drop  out  of  the  CARS.  Also,  the  total  amount  shown  for  the 
program  in  the  OSD  budget  may  include  other  contracts  not  reported  in  CARS. 

Due  to  the  data  difficulties  described  above,  we  were  unable  to  test  our  first  hypothesis 
using  our  selected  proxy  measure  for  transaction  costs.  In  fact,  it  seems  measuring  transaction 
costs  directly  or  by  proxy  from  the  existing  data  bases  may  not  be  possible. 

As  an  alternative,  we  could  always  infer  that  differences  due  to  complexity  or  technology 
maturity  imply  higher  transaction  costs  and  use  this  categorization  to  compare  programs  with 
assumed  high  transaction  costs  to  programs  with  assumed  low  transaction  costs.  For  example, 
Brown,  Flowe  and  Hamel  define  transaction  costs  as  costs  associated  with  interdependent 
activities,  and  they  suggest  that  joint  programs  (involving  two  or  more  services)  would  have 
higher  transaction  costs  than  single-service  programs.  Certainly,  the  coordination  costs  could 


29  The  name  of  this  program  changed  to  Expeditionary  Fighting  Vehicle  (EFV)  in  FY03. 
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be  assumed  to  be  higher  for  joint  programs.  When  they  examined  84  DoD  weapon  system 
programs  (45  joint  and  39  single)  in  terms  of  number  and  type  of  programmatic  breaches,  they 
found  that  joint  programs  were  statistically  more  likely  to  experience  breaches  in  schedule, 
research  development  testing  and  evaluation  costs  and  unit  costs.  If  we  assume  joint  programs 
have  higher  transactions  costs,  then  their  findings  may  suggest  that  programs  with  higher 
transactions  costs  require  distinct  measures  and  metrics  to  improve  cost  and  schedule 
estimates. 

IX.  Summary  and  Policy  Recommendations 

Developing  accurate  cost  forecasts  is  essential  for  future  budgeting  and  to  improving 
defense  decision-making.  This  paper  has  documented  a  growing  body  of  evidence  of  a 
systematic  downward  bias  in  initial  cost  estimates  for  major  weapon  systems.  An  important 
consequence  of  these  forecasting  difficulties  is  that  “major  weapon  systems  [. . .]  are 
experiencing  recurring  problems  with  cost  overruns,  missed  deadlines,  and  performance 
shortfalls”  (GAO,  2006,  March,  p.1).  Runaway  costs  threaten  to  weaken  the  armed  forces. 
Cash-flow  shortfalls  lead  to  quantity  reductions,  reprogramming,  and  funding  reductions  for 
other  programs.  These  cost  increases  mean  that  the  DoD  cannot  produce  as  many  weapons  as 
intended  nor  deliver  those  weapons  to  the  warfighter  when  promised.  Cost  overruns  squeeze 
existing  and  competing  programs.  Higher  price  tags  mean  that  fewer  of  those  weapons  end  up 
in  the  hands  of  soldiers,  and  that  options  on  new  weapons  cannot  be  exercised  without  breaking 
the  budget. 

Two  factors  are  usually  blamed  for  unrealistically  low  cost  forecasts:  bad  incentives 
(psychological  and  political-economic  explanations)  and  bad  estimates  (methodological 
explanations).  The  focus  of  this  paper  is  on  cost  methodology.  The  case  studies  suggest  that 
inclusion  of  transaction  costs  as  a  recognized  component  of  total  cost  adds  to  explanatory  and 
forecasting  power  of  cost-estimation  methods.  However,  we  also  found  that  current  data  bases 
are  not  well  structured  for  identifying  the  transaction  costs  of  major  acquisition  programs — for  a 
number  of  reasons. 

What  can  be  done  to  reduce  actual  costs/expenditures?  TCE  suggests  identifying 
strategies  to  cut  coordination  and  motivation  costs.  Specific  recommendations  to  cut 
Coordination  and  Motivation  Costs  would  include  the  following: 

a.  Reducing  Complexity:  Investing  in  a  more  complete  contract  (increasing  search  and 
information  costs)  and  the  use  of  more  mature  technologies. 

b.  Reducing  Uncertainty  through  multi-year  contracts  (demand  uncertainty);  investing  in 
more  complete  contracts  (relationship  uncertainty). 

c.  Increasing  measurement  and  monitoring  to  reduce  information  asymmetries  (and 
associated  risks). 

d.  Putting  credible  deterrents  to  bad  behavior  in  place — such  as  penalty  clauses, 
warranties  and  bonding;  using  multi-year  contracts  to  gather  information  and  reward 
good  reputations. 

e.  Mitigating  the  uncertainties  introduced  by  asset  specificity  through  careful  use  of 
incentives,  proper  bundling  of  goods  and  services  and  GOCO  assets. 


n.-.i'.uMiAftr.  v  hsiu,, 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  530- 


f.  Increasing  contestability  of  government  contracts  through  government-controlled 
standby  capacity  (threat  of  vertical  integration),  second  sourcing,  and  preservation  of 
real  options  (threat  of  entry  of  competing  suppliers). 

That  said,  however,  the  primary  insight  drawn  from  TCE  is  that  correctly  estimating 
production  costs  is  necessary,  but  not  sufficient.  The  choice  of  contract,  organization,  and 
incentives — along  with  key  characteristics  of  markets  and  transactions  (uncertainty,  complexity, 
asset  specificity,  frequency,  and  contestability) — must  be  included  to  obtain  reliable  cost 
estimates.  Leveraging  TCE  in  this  way  could  help  characterize,  explain,  forecast,  and  ultimately 
reduce  the  cost  growth  that  plagues  many  of  today’s  major  investments  in  military  capabilities. 
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Introduction 

The  Department  of  Defense  (DoD)  has  an  annual  budget  approaching  a  half  trillion 
dollars.  A  significant  portion  of  that  budget  is  either  directly  or  indirectly  affected  by 
Information  Technology  (IT)  Infrastructure  modernization  initiatives.  Nationally,  investment  in 
IT  infrastructure  modernization  is  about  $250  Billion  a  year,  spanning  approximately  175,000 
projects.  Unfortunately,  various  studies  indicate  that  only  28%  are  completed  on  time  and  on 
budget — with  the  number  dropping  to  around  9%  for  larger  companies  in  1994,  including 
government  programs.  By  1998,  these  numbers  had  improved,  with  success  rates  for  larger 
companies,  for  example,  up  to  24%.  But,  only  small  organizations  have  managed  to 
implement  more  than  half  of  their  applications  into  one  integrated  system  (Smith,  2000; 
Johnson,  1999,  December;  Keller,  2006,  May  29). 

The  factors  driving  these  numbers  are  myriad — and  include  the  sheer  scope  and 
complexity  of  infrastructure  modernization  programs,  unstable/undefined  requirements, 
unstable  funding,  moving  target  objectives,  and  evolving  threats.  As  a  result,  the  costs  of 
integration  complexity  increase  exponentially,  but  yet  are  almost  invariably  under  estimated. 
These  challenges  notwithstanding,  one  Gartner  study  asserts  that  IT  asset  productivity  will 
drive  market  capitalization  (Gartner,  2002,  July). 

Given  the  scope,  importance  and  complexity  of  these  projects,  reliable,  cost- 
effective,  early  warning  indicators  of  problems  are  essential.  Yet,  classical  investment  theory 
provides  little  guidance  for  dealing  with  public-sector  investment.  The  result  is  a  general 
absence  of  computationally  efficient,  predictive  models  applicable  to  the  analysis  of  those 
investments. 

However,  there  is  progress  on  several  fronts.  At  the  micro  level,  Earned  Value 
Management  is  gaining  acceptance.  But,  more  comprehensive,  flexible  methods  can  be 
developed  by  viewing  a  Firm  as  an  engineered  artifact  whose  responses  to  a  range  of  inputs 
can  be  characterized  in  terms  of  duration,  mass,  time,  stability,  and  location.  (By  way  of 
terminology,  a  “Firm”  refers  to  both  public  and  private  organizations  with  investment 
responsibilities.) 

How  well  a  Firm  executes  an  investment  depends  largely  on  how  well  it  acquires  and 
uses  information.  The  efficiency  of  that  use  provides  a  basis  for  pricing  the  value  of  a  Firm, 
independent  of  valuations  derived  from  equity  markets. 
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For  the  private  sector,  Capital  Asset  Pricing  Models  (CAPM)  models  provide 
valuation  tools,  of  which  the  Black-Scholes  equation  is  the  most  well  known  and  successful 
example.  These  models  are  based  on  the  observation  that  equity  markets  have  properties 
that  can  be  analyzed  using  models  based  on  the  “Law  of  Large  Numbers.” 

Unfortunately,  the  conditions  enabling  the  quantitative  analysis  of  private-sector 
investments  do  not  apply  in  the  public  sector  because  there  is  no  market  for  public-sector 
goods.  Nonetheless,  all  Firms,  regardless  of  whether  they  are  in  the  private  or  public  sector, 
must  respond  to  a  range  of  random  (external  and  internal)  perturbations.  The  responses  can 
be  accurately  modeled  using  mathematical  processes  based  on  the  “Law  of  Large 
Numbers.”  In  particular,  the  operation  of  a  Firm  can  be  modeled  as  a  stochastic  feedback 
system  using  algorithms  from  Information  Theory  and  System  Control  Theory. 

IT  Infrastructure  Investments — in  the  Public  Sector 

Private  equity  markets  provide  a  profit  incentive  to  resolve  uncertainty,  which  has  as 
one  of  its  effects  the  rapid  aggregation  of  information  from  large  numbers  of  participants.  In 
general,  the  larger  and  more  diverse  the  number  of  sources,  the  more  accurate  is  a  Firm’s 
valuation.  (With  the  Web  and  the  Internet  dramatically  lowering  the  cost  of  information, 
similar  aggregation  effects  are  occurring  in  the  news  media,  entertainment,  and  politics,  as 
evidenced  by  the  impact  of  Napster,  U-Tube,  Google,  etc.) 

Unfortunately,  the  Public  Sector  lacks  an  incentive  mechanism  and  usually  consists 
of  a  few  service/product  providers,  and  generally  only  one  customer — the  government. 
(Markets  with  few  sellers  and  buyers  are  more  suited  to  analysis  by  Game  Theory.) 

Despite  these  differences,  private  and  public  sectors  share  a  range  of  common 
concerns,  especially  for  large  scale  IT  modernization  projects.  Among  these  is  a  compelling 
need  to  answer  questions  such  as: 

■  What  level  of  uncertainty  surrounds  cost/schedule  estimates,  especially  at  the  onset 
of  a  major  investment  in  IT  infra-structure? 

■  Under  what  conditions  will  risk/uncertainty  decline  or  rise?  And  at  what  rate? 

■  How  can  risk/uncertainty  concerning  cost,  schedule,  and  scope  be  identified  in  a 
timely  manner,  quantified,  and  mitigated? 

■  How  can  a  Firm  best  respond  to  disruptions  to  supply,  budget  and  schedule  or  to  the 
introduction  of  new  technologies  by  competitors? 

■  At  what  point  in  a  project-development  cycle  will  estimates  of  cost,  schedule,  and 
scope  become  both  stable  and  credible? 

■  How  can  confidence  levels  in  cost  and  schedule  estimates  be  measured? 

■  Are  project  requirements  under  active  management  sufficiently  stable  to  ensure 
project  completion  on  time  and  within  budget? 

■  What  (quantitative)  models  can  be  used  to  determine  the  maximum  effective  rate  of 
investment  for  public-sector  projects? 

■  How  can  an  optimal  investment  portfolio  be  constructed? 

■  And,  how  is  such  a  portfolio  optimality  measured? 
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■  How  do  the  performance  measures  such  as  requirements  stability,  work  package 
completion,  rework  rates  and  resolution  rates  for  major  and  minor  issues  correlate 
with  each  other? 

■  Is  there  a  level  of  disruption  or  tipping  point  to  the  schedule,  cost,  and  resource 
allocations  beyond  which  a  project  cannot  recover? 

Developing  a  quantitative  framework  capable  of  answering  these  questions  is  the 
objective  of  the  paper.  The  strategy  is  to  employ  key  parameters  governing  the  efficiency  of 
a  Firm’s  operation  in  terms  of  observable,  easy-to-compute  variables  in  mathematical 
models  of  the  underlying  process  dynamics. 

Portfolio  Investment  Management — An  Overview 

Large-scale  Defense  infrastructure  modernization  programs  such  as  Global  Combat 
Support  have  complex  inter-dependencies  and  long-time  horizons  that  render  fully  informed 
investment  decisions  difficult  to  achieve  before  substantial,  and  unrecoverable,  resources 
are  committed.  However  complex  these  decisions,  they,  nonetheless,  can  be  decomposed 
along  three  basic  dimensions: 

■  Uncertainty 

■  Timing 

■  Irreversibility 

These  primary  parameters  define  the  value  of  investment  options  available  to  a  Firm, 
regardless  of  whether  it  is  in  the  public  or  private  sector.  Unfortunately,  algorithms  capable 
of  modeling  the  effects  of  these  variables  are  relatively  few,  especially  for  the  uncertainty 
and  irreversibility  of  investment  decisions  (Dixit  &  Pyndik,  1994,  p.  211).  For  large-scale 
Information  Technology  (IT)  modernization  programs,  there  are  at  least  three  sources  of 
uncertainty — and,  thus,  risk 

■  The  technical  complexity 

■  The  programmatic  complexity  of  integrating  software  intensive  systems 

■  The  absence  of  accurate  cost  information  at  the  onset  of  major  systems/  software 
programs 

Software-intensive  systems  are  particularly  sensitive  to  the  systematic  under¬ 
estimation  of  risk,  primarily  because  the  level  of  complexity  is  hard  to  manage,  let  alone 
comprehend.  Investment  in  software-intensive  systems  tends  to  be  irreversible  because  it  is 
spent  on  the  labor  required  to  develop  the  intellectual  capital  embedded  in  software. 

The  outcome  of  software  development  is  almost  invariably  unique,  a  one-of-a-kind 
artifact — despite  the  numerous  efforts  to  develop  reusable  software.  Unlike  physical  assets, 
the  salvage  value  of  software  is  zero  because  no  benefit  is  realized  until  the  system  is 
deployed;  and  that  labor,  once  invested,  is  unrecoverable.  One  result  is  an  (implicit) 
incentive  to  continue  projects  that  have  little  chance  of  success,  despite  significant  cost 
overruns,  schedule  delays. 

Indeed,  an  analysis  of  several  hundred  NASA  projects  indicates  that  accurate 
estimates  at  project  onset  are  virtually  impossible  to  achieve,  which  raises  concern  for  the 
validity  of  initial  Planned  Value  estimates  since  they  are  the  basis  of  Earned  Value 
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calculations  (Suter,  2005,  p.  261).  Thus,  measures  of  uncertainty  for  cost/schedule 
estimates  and  the  rate  at  which  that  uncertainty  declines  are  a  key  concern — because,  they 
govern  whether  and  to  what  extent  confidence  can  be  placed  in  cost  and  schedule 
estimates.  The  key  to  overcoming  initial  estimate  uncertainty  is  the  capability  to  harness  and 
to  apply  information  as  it  becomes  available,  thus,  enabling  a  Firm  to  capture  the  time  value 
of  that  information. 

Indeed,  where  IT  infrastructure  modernization  projects  are  supported  by  a  strong 
quality-assurance,  systems-engineering  culture  (e.g.,  have  institutionalized  best-practice 
regimes  such  as  the  CMMI,  6-Sigma,  Agile  Methods)  are  likely  to  quickly  reduce  estimate 
errors  incurred  at  project  start-up.  Firms  without  that  culture  tend  to  have  limited  information 
efficiency.  (Drawing  an  analogy  to  thermo-dynamic  systems,  such  Firms  constitute  highly 
dissipative  systems  in  that  they  exhibit  a  high  degree  of  entropy,  which  takes  the  form  of 
information  disorganization). 

Unfortunately,  traditional  methods  of  discounting  investment  risk  such  as  Net  Present 
Value  (NPV)  do  not  account  for  irreversibility  and  uncertainty.  In  part,  this  is  due  to  the  fact 
that  NPV  computes  the  value  of  a  portfolio  of  investments  as  the  maximized  mean  of 
discounted  cash  flows  on  the  assumption  that  the  risk  to  underlying  investment  options  can 
be  replicated  by  assets  in  a  financial  market. 

NPV  also  implicitly  assumes  that  the  value  of  the  underlying  asset  is  known  and 
accurate  at  the  time  the  investment  decision  is  made. 

These  assumptions  seldom  apply  for  large-scale  infra-modernization  programs,  in 
either  the  public  or  the  private  sector.  In  addition,  NPV  investment  is  undertaken  when  the 
value  of  a  unit  of  capital  is  at  least  as  large  as  its  purchase  and  installation  costs.  But,  this 
can  be  error  prone  since  opportunity  costs  are  highly  sensitive  to  the  uncertainty 
surrounding  the  future  value  of  the  project  due  to  factors  such  as  the  riskiness  of  future  cash 
flows.  These  considerations  also  extend  to  econometric  models,  which  exclude 
irreversibility,  the  incorporation  of  which  transforms  investment  models  into  non-linear 
equations  (Dixit  &  Pindyck,  1994,  p.  421).  Nonetheless,  irreversibility  constitutes  both  a 
negative  opportunity  cost  and  a  lost-option  value  that  must  be  included  in  the  cost  of 
investment. 

In  addition,  the  competitive  equilibrium  of  a  market  is  virtually  never  stationary,  even 
in  the  long  run.  Rather,  it  is  a  dynamic  process  in  which  prices  can  fluctuate  widely,  and, 
thus,  contribute  to  uncertainty.  Neither  classical  investment  theory  nor  discounting  methods 
such  as  NPV  take  these  factors  into  account.  Yet,  for  long-term,  capital-intensive 
investments  such  as  oil  exploration  and  IT  infrastructure  modernization,  price  fluctuation 
constitutes  significant  risk — which  must  be  factored  into  investment  decisions  (1994,  p.  396). 

These  difficulties  are  due  to  an  underlying  limitation  common  to  both  classical 
investment  theory  and  valuation  methods  such  as  NPV:  their  reliance  on  simple  equilibrium 
relationships  between  rates  of  investment  and  risk,  which  has  the  practical  effect  of 
precluding  the  effect  of  uncertainty  and  irreversibility  on  investment.  The  factors  that 
adversely  impact  NPV  also  impact  the  accuracy  of  Planned  Value  benchmarks  that  are  the 
basis  of  Earned  Value  Calculations  (Suter,  2006,  p.  406).  For  these  reasons,  ’’classical” 
methods  have  met  with  little  success  in  providing  accurate  valuations  of  Public  investments 
and  qualified  success  for  those  in  the  Private  Sector. 
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Capital  Asset  Pricing  models  such  as  Black-Scholes,  however,  sidestep  these 
problems  by  transforming  the  analysis  from  a  deterministic  formulation  to  one  based  on 
probability.  To  handle  uncertainty  and  risk  driven  by  price  fluctuations,  it  uses  the  Ito  Lemma 
to  compute  valuations.  Black-Scholes  proceeds  from  the  assumption  that  there  is  a  true 
value  for  a  stock  that  corresponds  to  its  risk,  and  that  value  can  be  used  to  decide  whether 
the  market  price  for  a  stock  is  too  high  or  too  low.  That  is,  an  option’s  value  equals  the  value 
of  the  information  concerning  that  risk. 

Black-Scholes  models  the  price  of  a  stock  option  as  a  Market-driven  process  defined 
by  Eqn  [3.1],  the  fundamental  condition  of  equilibrium  (Dixit  &  Pindyck,  1994,  p.  115;  Cover 
&  Thomas,  1991 ,  p.  28): 

[3.1]  p  =  r  +  9*ct  *  Pxm 

Where: 

p  =  the  risk  adjusted  return 

0  =  market  price  of  risk 

r  =  riskless  rate  of  return 

Pxm  =  coefficient  of  correlation  between  returns  on  the  particular  asset  (“x”  subscript) 
and  the  entire  market  portfolio  of  stocks  (denoted  by  the  “m”  subscript). 

a  =  the  proportional  variance  parameter 

dz  =  the  increment  of  the  standard  Wiener  process 

The  computational  efficiency  of  Black-Scholes  model  enables  Floor  Traders  (and 
computer-programmed  trading  algorithms)  to  exploit  small,  short-term  price  fluctuations  in 
real  time  and  to  use  new  pricing  information  to  continually  rebalance  portfolios.  Investors  (as 
distinguished  from  Floor  Traders  by  virtue  of  their  longer-term  time  horizon)  find  Black- 
Scholes  no  less  useful  because  of  its  ability  to  link  risk  to  valuation  over  timeframes  ranging 
into  years.  More  recent  refinements  such  as  Levy  processes  introduce  more  realism  by 
generalizing  Brownian  motion  processes  to  include  discrete  state  jumps.  (The  jumps  can  be 
local,  global,  simultaneous,  independent  or  correlated.) 

The  significance  of  Black-Scholes  is  its  computational  efficiency  for  modeling  price, 
interest,  and  discount  rates — using  a  few  readily  observable  parameters  that  provide 
reasonable  approximations  to  the  underlying  physical  processes  using  methods  based  on 
the  Law  of  Large  Numbers.  In  particular,  it  specifically  eschews  unobservable/hard-to- 
measure  parameters  such  as  “investor  psychology.”  The  Black-Scholes  strategy  is  at 
variance  with  efforts  to  improve  the  analysis  by  increasing  the  dimensionality  of  the  problem 
via  the  addition  of  more  parameters  (as  is  often  done  with  econometric  models,  and  on 
occasion  with  Balance  Scorecard  methods).  The  addition  of  extra  parameters  may  well 
provide  a  more  detailed  picture  of  performance,  but  at  additional  cost  and  without 
necessarily  improving  the  accuracy  needed  for  decision-making  (McShea,  2006,  November, 
p.  31).  As  a  result,  these  refinements  have  added  relatively  minimal  value  and  can  be 
relatively  expensive  to  implement. 
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The  key  is  to  recognize  that  decision-making  is  not  necessarily  dependent  on  a 
detailed  understanding  of  causality  to  be  effective.  Thus,  Capital  Asset  Pricing  models  focus 
on  those  few  variables  with  the  most  explanatory  power.  The  objective  is  not  to  predict 
which  firms  are  most  likely  to  succeed,  but  only  what  they  are  “worth”  as  measured  against 
various  combinations  of  risk,  uncertainty,  interest  rates,  and  competing  investment 
opportunities.  The  task  of  this  paper  is  to  provide  equivalent,  computationally  efficient 
methods  for  estimating  valuations  in  public-sector  investments,  using  the  fewest  parameters 
with  the  most  predictive  power. 

There  are,  of  course,  methods  other  than  market-based  Capital  Asset  Pricing  for 
determining  asset  valuations.  But,  these  also  suffer  from  various  limitations.  For  example, 
Dynamic  Programming  (DP)  could  be  applied  to  public-sector  investments  and  is  useful  in 
solving  multi-stage  optimization  problems,  but  only  if  a  small  number  of  possible  choices 
exist  at  each  stage.  Indeed,  a  small  increase  in  the  number  of  possible  choices  leads  to  a 
combinatorial  explosion,  thus  curtailing  overall  efficiency  of  DP. 

Another  method  is  the  Discrete  Binomial  Model,  which  uses  a  risk-adjusted 
stochastic  process  for  modeling  the  underlying  asset.  The  strategy  is  to  approximate 
uncorrelated  investment  dynamics  using  two  basic  ideas:  a  change  of  time  scale  and  a 
change  of  the  basis  of  the  asset  span  to  approximate  uncorrelated  geometric  Brownian 
motion.  Yet  another  variant,  the  Lattice  Binomial  model,  has  proved  useful  for  valuing 
complex  option  problems  when  payoff  depends  on  multiple  state  variables  that  follow 
correlated  geometric  Brownian  processes.  In  this  case,  the  strategy  is  to  approximate  a 
multi-dimensional  geometric  Brownian  motion  with  a  binomial  lattice  by  choosing  the  size 
and  the  probabilities  of  the  jumps  so  that  the  characteristic  function  of  the  discrete 
distribution  converges  to  the  characteristic  function  of  the  continuous  distribution. 

But,  both  methods  require  knowledge  of  the  underlying  probability  distributions — a 
requirement  that  can  be  difficult  to  satisfy — and  will  converge  to  a  solution  only  in  the  limit. 
For  practical  applications,  the  time  required  to  acquire  sufficient  data  to  identify  a  convergent 
solution  can  preclude  widespread  application,  especially  if  the  time  value  of  information 
rapidly  declines,  thus  forcing  the  decision-maker  to  decide  on  acting  with  incomplete 
information,  or  on  risking  being  overtaken  by  events. 

Information  Theory  and  “Synthetic  Prices” 

In  competitive  markets,  a  single  number — the  price  of  a  Firm’s  stock — represents 
risk.  Under  ideal  conditions  that  price  fully  captures  the  Firm’s  internal  efficiencies,  Return- 
on-lnvestment  (ROI)  and  earning  potential.  Those  efficiencies  determine  the  Firm’s 
capability  to  harness  new  information  as  it  becomes  available.  For  private-sector  firms, 
prices  provide  two  important  types  of  information: 

1.  The  rate  at  which  information  becomes  known. 

The  rate  is  analogous  to  the  diffusion  problem  in  heat  transfer,  which  means  that 
information  diffusion  can  be  modeled  as  a  Brownian  motion  processes. 

2.  How  information  is  aggregated. 

In  the  private  sector,  the  efficiency  of  aggregation  indicates  market  efficiency;  yet, 
while  no  such  aggregation  occurs  in  public-sector  markets  because  there  is  no  incentive. 
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Hayek  was  the  first  to  identify  these  effects  and  to  provide  the  rationale  for  defining 
markets  in  terms  of  their  information  value.  The  definition,  in  highly  abbreviated  form,  is  as 
follows: 


Competitive  markets  provide  for  the  efficient  coordination  of  decisions  involving  time 
and  uncertainty.  The  process  can  be  modeled  as  a  Random  Walk  (in  which  the  limit 
is  approximated  by  a  Brownian  motion  process).  The  rationale  derives  from  the  fact 
that  where  information  flows  without  impediments,  stock  prices  immediately  reflect 
the  latest  information — so  that  a  price  change  today  will  reflect  only  today’s  news  and 
is  independent  of  any  prior  price  change  (such  processes  are  a  weak  form  of  market 
efficiency  and  have  been  modeled  with  some  success  using  Markov  models)  (Dixit  & 
Pindyck,  1994,  p.  63).  And,  since  news  often  is  unpredictable,  price  changes  are  also 
unpredictable,  but  fully  reflect  all  known  information — thus,  justifying  the  Random 
Walk  interpretation. 

However,  competitive  markets  are  not  the  only  mechanism  for  determining 
investment  valuations.  A  Firm’s  internal  (information)  processing  efficiencies  enable  it  to 
reallocate  resources  based  on  new  information — and,  thus,  to  manage  risk.  The  efficiency  of 
that  process  can  be  measured  independently  of  market-based  valuation.  In  this  way,  it  can 
be  applied  to  public-sector  Firms  to  construct  “synthetic  prices” — thus  linking  valuations  of  its 
investments  to  its  asset  of  a  Firm:  the  ability  to  process  information  efficiently. 

For  the  private  sector,  perturbation-based  measures  should  converge  in  the  limit  to 
the  market-based  valuation,  thus  providing  a  basis  for  testing  the  validity  of  using  the 
internal  efficiencies  to  derive  a  “synthetic  price”  for  the  value  of  a  Firm  (Hayek,  945, 
September,  p.  35). 

In  fact,  the  latter  should  reflect  more  accurately  the  “true”  value  of  a  Firm,  which  is 
what  “Value  Investors”  and  Hedge  Funds  are  constantly  trying  to  identify.  Indeed, 
perturbation-response  models  offer  a  means  to  quantitatively  link  micro,  Firm-level  models 
to  macro,  policy-level  models. 

Pricing  Public-sector  Investments 

Measures  of  investment  efficiency  for  the  public  sector  enable  a  portfolio  manager  to 
keep  investments  allocated  to  the  most  profitable  outcomes — despite  shocks  and 
perturbations  to  operations.  But,  effective  Portfolio  Management  depends  on  accurate 
pricing  information.  When  information  is  limited  or  uncertain,  risk  is  not  efficiently  priced.  As 
a  result,  the  marginal  social  utility  of  an  investment  will  not  equate  to  its  price — thus,  leaving 
no  means  to  identify  a  “socially  optimal”  level  of  investment  in  either  the  private  or  the  public 
sector.  The  net  effect  is  analogous  to  Nash  Equilibrium:  a  sup-optimal  investment 
equilibrium  condition  that  occurs  when  no  player  has  incentive  to  unilaterally  change 
strategy — because  a  change  by  any  one  of  them  would  lead  that  player  to  earn  less  than 
would  be  obtained  by  remaining  with  the  current  strategy  (Cover  &  Thomas,  1991 ,  p.  460, 
475;  Dixit  &  Pindyck,  1994,  p.  147,  283). 

Sub-optimality  can  be  driven  by  limited  information,  by  a  lack  of  incentives  to  change 
(as  is  the  case  for  the  Nash  Equilibrium),  as  well  as  by  uncertainty  driven  by  factors  such  as 
market  volatility.  Higher  volatility  estimates  reflect  greater-than-expected  fluctuations  in 
underlying  price  levels  and  result  in  higher-option  premiums  for  both  puts  and  calls.  With 
respect  to  the  internal  processing  efficiencies  of  a  Firm,  the  Taguchi’s  common  and  special 
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sources  of  variability  drive  the  perturbations  of  a  Firm’s  workflow  (Leon,  Shoemaker,  & 
Kacker,  1987).  The  efficiency  of  response  determines  whether  and  to  what  extent  a  Firm 
can  synthesize  information  into  actionable  decisions  in  a  timely  manner.  The  often 
inconclusive,  interwoven,  and  ambiguous  nature  of  available  information  can  result  in  time 
consumed  to  assess  its  value,  which  depreciates  the  time  value  of  information.  The  resulting 
delay  propagates  uncertainty,  which  equates  to  opportunities  lost.  How  much  time  is 
required  to  resolve  the  ambiguity  is  a  function  of  a  Firm’s  ability  to  manage  that  variability, 
which  depends  on  its  ability  to  synthesize  information. 

The  impact  of  information  efficiency  can  be  measured  in  terms  of  response  to 
random  perturbations.  The  parameters  characterizing  the  perturbations  include  the 
amplitude  and  time  lag  for  changes  in  valuation  and  resource  allocations,  the  amount  of 
work/rework  completed,  requirements  churn,  the  ratio  of  assets  invested  to  return  on  those 
assets,  output  price/cost,  fluctuations  in  labor,  resource  availability,  the  time  to  close  major 
issues,  the  time  to  identify  and  to  minimize  both  special  and  common  causes  of  variability, 
schedule,  and  price,  etc.  Note  that  while  these  parameters  indicate  a  Firm’s  internal 
information  processing  efficiencies  with  respect  to  work  progress,  much  as  Earned  Value 
does,  they  say  nothing  of  business  value — i.e.,  whether  the  product  provides  anything  useful 
to  a  customer. 

The  perturbation-responses  are  governed  by  structure  of  a  Firm,  which  can  be 
represented  as  a  quasi-deterministic  State  Variable  model.  The  combination  of  these 
techniques  also  can  be  used  to  identify  the  boundaries  beyond  which  the  magnitude  of  the 
perturbations  would  result  in  unstable  responses  and  project  failure  (Ford  &  Taylor,  2006, 
pp.  337-369). 30  Information  Theory  benefits  the  analysis  through  the  provision  of  algorithms 
that  reduce  the  number  of  (perturbation-response)  states  to  be  considered  in  evaluating  a 
process.  The  larger  the  number  of  states,  the  greater  the  lack  of  specificity  or  uncertainty 
(entropy)  of  the  system.  The  following,  highly  over-simplified  example  illustrates  the  point. 

The  number  of  states  that  a  single  dice  can  assume  is  6.  But,  if  we  know  that  the 
dice  is  in  some  sense  biased  toward  either  even  or  odd  numbers  (i.e.,  we  have  additional  or 
“side”  information),  then  the  number  of  states  that  need  to  be  considered  is  effectively 
reduced  to  3,  thus  changing  the  outcome  probabilities  but  not  the  probability  distribution 
model.  Alternatively  stated,  the  entropy  (disorganization)  of  a  system  does  not  depend  on 
the  actual  values  taken  by  the  random  variables  describing  it,  but  only  on  the  associated 
probabilities. 

Changes  in  the  probability  space,  driven  by  changes  to  the  efficiency  of  information 
flows  within  a  Firm,  provide  a  quantitative  basis  for  linking  the  efficiency  of 
information/knowledge  management  to  well-defined  mathematical  processes,  thus  providing 
quantitative  measures  of  risk  that  correspond  to  the  underlying  physical  processes. 

Information  Theory  can  enable  decision-makers  to  reduce  risk,  often  drastically,  by 
providing  a  quantitative  framework  to  address  issues  such  as: 

1 .  The  quantitative  determination  of  changes  in  the  uncertainty  levels  associated  with 

cost/schedule/resource  estimates  as  a  project  proceeds  through  its  lifecycle. 


30  Ford  and  Taylor's  text  provides  detailed  discussion  of  project/program  stability. 
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2.  Discounting  the  anticipated  benefits  from  a  project — by  measuring  risk  as  a  function 
of  the  time  lags  and  amplitude  of  a  Firm’s  response  to  various  types  of  perturbations. 

3.  Identifying  and  reducing  (both  normal  and  special)  sources  of  variability  that 
adversely  impact  work  progress  and  service/product  quality 

4.  Constructing  rules  to  achieve  optimal  investments — in  terms  of  ensuring  that  a  Firm’s 
decision-making/information  is  efficient 

5.  The  use-state  variable  models  to  estimate  whether,  and  at  what  rate,  investment 
management  is  improving 

Models  capable  of  answering  these  questions  we  consider  next. 

Computational  Models 

The  Firm’s  responses  to  the  perturbations  provide  the  raw  data  from  which  (indirect) 
measures  of  that  efficiency  can  be  expressed  in  terms  of  factors  such  as  the  time  lag  and 
amplitude  of  responses,  and  the  variability  of  both.  The  perturbation/response  processes  are 
modeled  using  state  variable  regulator/  controller  design  methods.  One  of  the  best  known 
and  widely  applied  is  the  Kalman  Filter,  which  can  be  used  to  measure  how  uncertainty 
propagates  over  time,  and,  thus,  calculate  the  information-carrying  capacity  of  a  Firm. 

The  output  of  the  Kalman  Filter  model  is  the  amount  and  rate  of  information  gain 
produced  by  various  organizational  structures.  The  magnitude  and  rate  of  correction  serve 
as  measures  of  a  Firm’s  information  processing  capability,  and  (by  implication)  its  level  of 
entropy  or  level  of  internal-information  organizational  efficiency.  The  more  efficient  the  Firm, 
the  more  quickly  it  will  respond  to  perturbations  (random  shocks)  regardless  of  source, 
internal  or  external.  (The  situation  is  analogous  to  determining  the  bandwidth  of  a 
communications  system). 

Schematically,  the  Kalman  Filter  information  flow/computation  cycle  is  illustrated  in 
Figure  1 ,  below. 
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Kalman  Filter 


Z~(k+1) 


xA(k+1|k+1) 


Legend: 

x  is  the  vector  of  variables  comprising  the  system,  whose  state  is  to  be  estimated  over 
the  successive  time  (e.g.,  multi-stage  investment)  periods  k  =0,1,2,... 

x  A  (k+1  |k)  is  the  predicted  estimate  of  x  for  time  “k+1”  based  on  measures  taken  at  time 
“k.” 

z  is  the  actual,  uncorrected  measurement  of  x. 

z  ~  is  estimate  of  x  when  corrected  errors  are  introduced  by  the  measurement  process, 
z  A  is  the  estimate  of  x  A  as  filtered  by  H. 

H  is  the  measurement  transformation  matrix  that  relates  the  system  state  vector,  x,  to  its 
measure,  z. 

Delay  is  the  lag  in  system  response  to  a  stimulus.  Delay  is  inherent  to  an 
organization  because  information  cannot  be  gathered,  analyzed,  or  transmitted 
instantaneously.  Thus,  changes  in  the  environment  or  slips  in  schedule  may  or  may  not  be 
recognized  when  they  occur.  For  example,  decreases  in  data  quality  typically  generate 
increased  disruption  in  operation.  As  more  resources  are  shifted  to  fixing  and  correcting 
data  records,  the  rate  at  which  information  is  processed  decreases.  The  resulting 
inefficiency  generates  increased  correction  and  rework  rates,  along  with  increase  delays  in 
task  completion. 
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F  is  the  coefficient  matrix  of  the  state  variable  vector  x.  It  describes  the  input/output 
efficiencies  of  a  Firm  and  its  investments.  If  these  change  overtime,  then  F  (k)  =/=  F 
(k+1),  k=0,  1,2...  and  the  system  described  by  Eqn  (1),  (2),  below,  would  be  non-linear 
in  time  “t.” 

The  steps  presented  in  Figure  1  are  as  follows: 

1.  Begin  with  xA(0  |0)  =0,  for  k  =  0,  1,2...  Use  xA(k  |  k)  to  iteratively  compute 
xA  (k  +1  ,k  +1 )  given  z(k+1 ). 


Where: 


xA  (  k  |  k)  Is  the  estimate  propagated  forward  by  pre-multiplying  it  by  F(k+1,  k)  to 
give  the  predicted  estimate  xA(k+1  |k). 

2.  Pre-multiply  xA(k+1  |  k)  by  H  (m  +1)  to  compute  is  zA(k+1  |  k),  which  is  then 
subtracted  from  the  actual  measurement  z  (k+1). 

■  The  result  is  z~(k+1  |  k),  which  is  the  measurement  residual — that  is,  the 
difference  between  actual  and  predicted  estimates  at  time  (k+1). 

3.  The  residual,  z  ~,  is  pre-multiplied  by  K  (k+1 ),  the  Kalman  Filter  coefficient  matrix, 
which  is  added  to  xA(k+1  |  k). 

■  The  result  is  xA(k+1  |  k  +1 ),  which  is  the  estimate  of  x  at  time  (k+1 ),  given  the 
measures  updated  at  time  (k+1). 

4.  xA  (k+1  |  k  +1),  the  optimal  filtered  estimate,  is  stored  until  the  next  measurement  is 
made — at  which  time  the  cycle  is  repeated.  (This  is  the  only  data  that  needs  to  be 
stored  between  measures,  thus  saving  considerable  computer  storage  and  memory.) 

Note  that  the  product  K  (k+1)*z~(k+1  |  k)  is  the  correction  that  is  added  to  the 
predicted  estimate  xA(k+1  |k)  to  determine  the  filtered  estimate. 

The  optimal  filter  consists  of  the  model  of  the  dynamic  process  which  performs  the 
function  of  a  prediction  and  feedback  correction  scheme  in  which  the  gain-times-residual,  K 
(k+1)  *z~(k+1  |k],  per  Eqn  [3]  below  enters  the  model  as  the  forcing  function  u(  k)  (Meditch, 
1969,  p.182). 

How  all  this  works  is  illustrated  by  the  following  state-variable  model  for  which  the 
following  assumptions  are  made. 

■  Time  delay  decreases  the  volume  of  work  accomplished  per  unit  of  time. 

■  Time  delay  is  driven  by  unstable/poorly  managed  requirements,  funding 
instability,  etc.,  thus  acting  to  increase  the  amount  of  rework. 

■  In  this  example,  model  parameters  are  assumed  to  be  Gaussian  (Normally) 
Distributed,  with  side  information  (e.g.,  feedback)  entering  into  the  system  as  a 
sequence  of  predict-correct  actions. 

x(t)  is  the  vector  of  state  variables,  which  consists  of  the  two  elements: 
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x-t  (t)  =Volume  of  work/unit  of  time, 
x2  (t)  =  Volume  of  rework/unit  of  time. 
The  state  variable  model  is: 


[  1  ]  x’  =  F*  x  (t)  +  u(t) 

(where  x’  =  dx  (t)/dt)  =  [x-T,  x2’]  =  [dXi  (t)/dt,  x2’  =  dx2  (t)/dt] 

[2]  [Xl’]=  [1  1]  [ Xl(t)]  +[1  0][Ul(t)] 

[x2]  [0  1]  [ x2(t)]  [0  1][u2(t)] 

Verbally,  Eqn  [2]  states  that  the  rate  of  change  in  work  package  completion  is  the 
sum  of  the  current  workload  (xl )  +  rework  (x2)  +  the  arrival  of  new  work  (ul ).  The  rework 
rate  (x2’)  is  equal  to  the  sum  of  current  rework  (x2)  +  new  rework  (u2),  where  the  vector  u(t) 
is  governed  by  the  corrections  provided  by  Eqn  [3], 

The  control  vector  u(t)  =  [u^t),  u2(t)  ]  is  defined  in  terms  of  predictor/corrector 
parameters  as: 

[3]  u(t)  =  K(t)*[z  -  H*  xA] 

Eqn  [3]  computes  the  correction,  given  xA  from  the  Kalman  Filter  Eqn  [9],  below. 

[4]  e(t)  =  K  (t)*[  v  -  H  *(  x  -  x  P)] 

Eqn  [4]  computes  the  error  estimate  prior  to  measurement. 

Eqns  [3],  [4]  derive  from  the  linear  measurement  Model: 

[5]  z  =  H*  x  +  B  *  v 

To  keep  the  computation  simple,  H,  B  are  defined  as  identity  matrices: 

H  =  [1  0] 

[0  1] 

B  =  [1  0] 

[0  1] 

v  is  a  vector  of  random  measurement  errors  which  are  independent  of  the  state  x.  (v 
is  implemented  with  Monte  Carlo  simulation  input.) 

Eqn  [1]  and  Eqn  [2]  can  be  cast  into  Kalman  Filter  format  by  defining  the  following 
parameters: 

X  P  is  the  Estimate  of  the  system  state  prior  to  measurement,  as  defined  in  [7-P], 
below. 
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P  P  is  Covariance  of  the  system  state  prior  to  measurement,  as  defined  in  [7-P], 
below. 

x  A  is  the  Estimate  of  the  system  state  after  measurement,  as  defined  in  [7-A], 

PA  is  Covariance  of  the  system  state  after  measurement,  as  defined  in  [7-A], 

[6]  R  =  E[v*v’]  Is  the  mean  of  the  measurement  error  vector. 

[7-P]  P  p  =  E[(x  p  -  x)*(  x  p  -  x) T]  Is  the  error  covariance  matrix  prior  to 
measurement. 

[7-A]  P  a  =  E  [(x  a  -  x)*(x  a  -  x)T]  Is  the  error  covariance  matrix  after  measurement. 

[8]  x  a  =  x  p  +  P  p  *H  pT*R  p  _1(z  -  H*  x  p)  is  the  optimal  estimate. 

Eqn  [8]  is  a  model  of  Eqn  [1]  with  a  correction  term  that  is  proportional  to  the 
difference  between  the  actual  measurement  z-,  and  the  predicted  measurement  H*xj 

To  minimize  the  subscript  clutter  used  to  denote  the  before  and  after  calculations,  PA 
is  changed  to  P,  for  after,  and  Pp  to  M,  for  prior.  Thus,  indexing  Eqn  [8]  for  measures  over 
successive  discrete  time  periods  becomes  Eqn  [8’]. 

[8’]  xAi  =  x "  +K  j  (z  j  -  H  *  x i'),  where  i  =m,  m+1  .... 
for  multi-stage  investments. 

The  Gain  Matrix  for  the  Kalman  Filter,  for  time  periods  i  =  1 ,  2. . .  is 

[9]  K  j  =  P  j  *  H  i  T*R  j _1 

Eqn  [9]  can  be  interpreted  as  the  proportionality  matrix  or  the  ratio  between 

uncertainty  in  the  covariance  matrix  P  j ,  after  measurement  at  time  “i”  and  uncertainty  in  the 

measurements  R  j  Eqn  [6]  (which  can  be  interpreted  as  the  effectiveness  of  a  management 
reporting  system). 

The  propagation  of  uncertainty  in  the  discrete  time  system,  Eqn  [1 1],  below,  is  based 
on  the  computation  of: 

[10]  Pj  =  (M  j'1  +H  j t*R  j _1*H  j)'1 

[10’]  Pj  =  Mj-Mj*HjT*(Hj*Mj*HjT  +  R  j)'1  H  j  *  M  j 

[11]  M  j  +1  =  F  j  *P  j  *F  j T  +  G  j  *Q  j  *G  j T 

Eqn  [11]  reflects  the  balance  between  the  new  information-forcing  function, 
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G  j  *Q  j  *G  j 1 ;  and,  information  processing  efficiency,  F  j  *P  j  *F  j 1 .  (In  a  traditional 

physical  system,  F  j  would  represent  the  damping  efficiency  of  the  system. 
Collectively,  Eqn  [10],  [10’],  and  [11]  describe  the  propagation  of  the  covariance  of 

the  error  estimate,  which  are  independent  of  the  measurements  z  j. 

Eqn  [9]-[1 1]  Is  the  Kalman  Filter  for  a  multi-stage  process. 

Prediction  beyond  the  last  measurement  for  states  indexed  as  i  =  m  +1 ,  m+2,  ...  is 
given  by  Eqn  [12], 


[12]  x  (j+1)  -  F  (j)  *  x(i)  +  G  (j)  *U  (j) 


Figure  2,  below,  provides  a  heuristic  illustration  (via  the  use  of  “canned”  data)  of  the 
damping  out  effects  of  the  Kalman  Filter  (illustrated  by  the  magenta  colored  line),  the 
retardation  of  that  effect  induced  by  response  delay  (as  denoted  by  the  green  dashed  line), 
with  the  black  line  illustrating  the  impact  of  a  special  cause  of  variability,  such  as  the  failure 
of  an  integration  test,  a  reduction  in  funding,  a  major  equipment  failure,  etc.  The  blue 
dashed  line  (highly  exaggerated)  illustrates  “normal”  variability  causes,  which  might  include 
ambiguous  governance,  the  impact  on  products  and  services  due  to  aging  equipment,  but 
which  shows  gradual  improvement  overtime. 


Figure  2.  Perturbation-response  Output  from  Kalman  Filter 
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Summary/Next  Research  Steps 

The  strategy  and  models  outlined  in  this  brief  paper  indicate  how  Information  Theory  can 
be  used  to  quantify  the  value  of  information  and  translate  that  value  into  a  “price”  for  public- 
sector  investments  that,  in  principle,  is  comparable  in  a  competitive  market. 

The  next  steps  include: 

■  Fully  defining  the  conditions  under  which  the  synthetic  prices  for  public-sector  goods  are 
comparable  in  competitive  markets. 

■  Developing  methods  to  normalize  comparisons  across  portfolios  of  diverse  investment 
projects. 

■  Applying  and  evaluating  the  models  using  “real”  data  from  public-  and  private-sector  IT 
infrastructure  investment  projects. 

■  Using  the  model  output  to  evaluate  “synthetic  prices”  based  on  information  gains  defined 
using  Kalman  Filters. 

■  Comparing  model  results  across  a  range  of  scenarios. 

■  Identifying  algorithm  improvements  that  accelerate  convergence  to  specific  price 
solutions. 

Glossary 


Term 

Description 

Black-Scholes  model 

A  mathematical  model  used  to  calculate  the  value  of  a  project 
or  an  investment  as  derived  from  market-place  dynamics, 
based  on  approximations  to  Brownian  motion  processes 

Brownian  Motion 

The  random  motion  of  particles  in  a  liquid.  The  mathematical 
model  of  describing  this  motion  is  the  Wiener  process.  A 
continuous  time  process  that  forms  the  basis  of  many 
important  mathematical  models  in  thermodynamics  and 

Finance 

Binomial  Lattice  Methods 

An  algorithm  for  valuing  complex  option  problems  whose 
payoff  depends  on  multiple  state  variables  following  correlated 
geometric  Brownian  processes 

Capital  Asset  Pricing  (CAP) 

The  concept  that  there  is  a  true  value  for  a  stock 
corresponding  to  its  risk,  for  which  various  computational 
models  can  be  used  to  determine  risk-adjusted  discount  rates 
for  investments,  and  to  decide  whether  a  stock  price  is  too 
high  or  too  low. 

Complexity 

There  are  at  least  two  basic  types  of  complexity: 

Descriptive  complexity  of  an  object — Kolmogorov  complexity. 
While  not  directly  computable,  it  can  be  bound  between 
computable  measures  to  describe  the  complexity  of  a 
sequence  of  symbols;  Computational  complexity — measures 
the  time  or  space  required  for  a  computation 

Derivative 

A  financial  instrument  that  derives  its  value  from  the  value  of 
some  other  financial  instrument  or  variable.  For  example,  a 
stock  option  is  a  derivative  because  it  derives  its  value  from 
the  value  of  a  stock.  An  interest-rate  swap  is  a  derivative 
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because  it  derives  its  value  from  one  or  more  interest  rate 
indices 

Dynamic  Hedging 

The  purchasing/selling  of  financial  instruments  to  reduce  or 
cancel  out  the  risk  in  another  investment  as  required  by 
changing  market  conditions 

Earned  Value 

Measures  the  dollar-value  work  completed  per  unit  of  time.  It 
is  a  measure  of  progress  against  an  objective,  from  which 
schedule  (SC)  and  cost  variances  (CV)  can  be  computed 
using  Planned  Value  (PV)  and  Actual  Cost  (AC). 

SV  =  EV  -  PV 

CV  =  EV-AC. 

But,  realistic  estimates  of  Planned  Value  are  seldom 
available — especially  at  the  on-set  of  a  large,  complex  project. 
Hence,  the  value  of  Information  Theory  lies  in  determining 
when  and  to  what  extent  confidence  can  be  placed  in  a 
benchmark  such  as  Planned  Value 

Efficient  Frontier 

A  concept  that  there  is  a  true  value  for  a  stock  corresponding 
to  its  risk;  this  theory  of  stock  price  is  called  Capital  Assets 
Pricing  Model  and  is  used  to  decide  whether  a  stock  price  is 
too  high  or  too  low. 

Entropy 

A  measure  of  the  disorganization  of  a  physical  system/The 
uncertainty  of  a  single  random  variable/The  minimum 
descriptive  complexity  of  a  random  variable.  An  irreducible 
level  of  complexity  below  which  a  signal  cannot  be 
compressed 

Equity  Markets 

A  (competitive)  stock  market  that  efficiently  coordinates 
decisions  involving  time  and  uncertainty 

Exogenous 

Refers  to  variables  whose  values  are  driven  by  factors 
external  to  the  firm,  or  processes,  of  interest 

Firm 

In  this  paper,  the  “Firm”  refers  to  an  organization  in  either  the 
private  or  public  sector  tasked  with  investing  in  and  developing 
new  products  and  services 

Game  Theory 

The  branch  of  applied  mathematics  and  economics  that 
studies  situations  where  players  choose  different  actions  in  an 
attempt  to  maximize  their  returns.  It  provides  a  formal, 
quantitative  modeling  approach  to  social  situations  in  which 
decision-makers  interact 

Information  Theory 

A  discipline  spanninq  mathematics,  economics,  phvsics, 
communication  theory,  statistics,  involving  the  quantification  of 
data.  For  communications,  the  goal  is  to  enable  as  much  data 
as  possible  to  be  reliably  stored  or  communicated  over  a 
channel. 

Ito’s  Lemma 

Is  used  to  integrate  and  differentiate  stochastic  processes.  An 
/to  process  can  represent  the  dynamics  of  the  value  of  a 
project  which  does  not  have  a  time  derivative  in  the 
conventional  sense-  because  its  fluctuations  over  (short) 
periods  of  time  do  not  have  derivatives 

Kalman  Filter 

A  mathematical  algorithm  that  operates  in  a  predict-correct 
fashion  that  uses  feedback  to  maintain  a  system  (e.g.,  a 
rocket,  or  an  investment  portfolio)  on  a  desired  trajectory 

Levy  Process 

A  generalization  of  Brownian  Motion  processes  to  include 
discrete  state  jumps.  The  jumps  can  be  local,  global, 

LUSU'>mritK!ivfn,1 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  548- 


simultaneous,  independent  or  correlated 

Law  of  Large  Numbers 

The  sum  of  independent,  identically  distributed  random 
variables  that  can  be  approximated  arbitrarily  closely  to  the 
expected  value  of  the  random  variables 

Markov  Property 

The  property  of  a  process  that  current  information  is  useful  for 
forecasting  the  future  path  of  a  process.  Applied  to  Stock 
processes  on  the  premise  that  public  information  is  quickly 
incorporated  into  the  current  price  /D-63/ 

Mutual  information 

The  communication  rate  (efficiency)  in  the  presence  of  noise. 

It  is  a  measure  of  the  amount  of  information  that  one  random 
variable  contains  about  another  random  variable.  It  is  the 
reduction  in  the  uncertainty  of  one  random  variable  induced  by 
knowledge  of  the  other 

Nash  Equilibrium 

A  condition  in  Game  Theory  in  which  no  player  has  incentive 
to  unilaterally  change  her  action.  Players  are  in  equilibrium  if  a 
change  in  strategies  by  any  one  of  them  would  lead  that 
player  to  earn  less  than  if  she  remained  with  her  current 
strategy 

Net  Present  Value  (NPV) 

NPV  is  a  standard  method  for  the  financial  evaluation  of  a 
lonq-term  project.  Used  for  capital  budqetinq,  and  widely 
throuqhout  economics,  it  measures  the  excess  or  shortfall  of 
cash  flows  (in  present  value  (PV)  terms)  once  financinq 
charges  are  met.  By  definition,  NPV  =  Present  value  of  net 
cash  flows 

Options  Pricing 

Is  a  contract  between  a  buyer  and  a  seller,  or  a  provision  of  a 
contract,  that  gives  one  party  (the  option  holder)  the  right,  but 
not  the  obligation,  to  perform  a  specified  transaction  with 
another  party  (the  option  issuer  or  option  writer)  according  to 
specified  terms.  Option  contracts  are  a  form  of  derivative 
instrument 

Portfolio  Management 

The  discipline  of  managing  a  portfolio  of  investments  with  the 
objective  of  maximizing  the  value  of  the  entire  portfolio  by 
reallocating  resources  among  the  investments  comprising  the 
portfolio 

Public-sector  Firms 

Firms  such  as  public  health  or  security  tasked  with  providing 
goods  and  services  whose  valuations  are  not  saleable  in 
private  equity  (i.e. ,  stock)  markets,  but  which  benefit  society 

Random  Walk 

A  process  that  takes  a  discrete  move  in  a  specific  direction 
according  to  a  specified  probability  distribution.  The  Brownian 
Motion  (Weiner  process)  is  the  limit  for  the  discrete  random 
walk  process 

Rate  Distortion  Theory 

A  maior  branch  of  information  theory;  it  addresses  the  problem 
of  determininq  the  minimal  amount  of  entropv  (or  information) 
that  can  be  communicated  over  a  channel,  so  that  the  source 
(input  signal)  can  be  approximately  reconstructed  at  the 
receiver  (output  signal)  without  exceeding  a  given  distortion 
level. 

Risk 

“Risk"  is  randomness  with  knowable  probabilities; 

"uncertainty"  is  randomness  with  unknowable  probabilities. 
Frank  Kniqht  (1921):  An  enqineerinq  definition  of  risk  is  “the 
(probability  of  an  adverse  event)  x  (loss  per  event)” 

ROI — Return  on  Investment 

A  measure  of  the  net  income  a  firm  is  able  to  earn  with  its  total 
assets.  Return  on  investment  is  calculated  by  dividing  net 

Lusu'>mruxiivfn,1 
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profits  after  taxes  by  total  assets.  The  Rate  of  Return  (ROR) 
or  Return  on  Investment  (ROI),  or  sometimes  just  return,  is  the 
ratio  of  money  qained  or  lost  on  an  investment  relative  to  the 
amount  of  money  invested.  The  amount  of  money  gained  or 
lost  mav  be  referred  to  as  interest,  profit/loss,  qain/loss,  or  net 
income/loss.  The  money  invested  may  be  referred  to  as  the 
asset,  capital,  Drincioal,  or  the  cost  basis  of  the  investment. 

Side  information 

Information  that  is  relevant  to  the  outcome  of  an  event  such  as 
a  coin  toss,  or  a  horse  race 

System  Control  Theory 

The  discipline  of  controlling  complex  machines  such  as 
aircraft,  computer  networks,  financial  and  manufacturing 
systems 

Uncertainty 

A  characteristic  of  a  random  variable  that  is  measured  as 
Entropy.  It  is  the  number  of  bits  required  to  describe  a 

Random  Variable.  The  larger  the  number  of  values  that  the 
random  variable  can  take,  the  larger  the  uncertainty. 

Variability: 

Special  and  common  sources 

A  common  source  of  variability:  The  quality  level  of  an  item 
created  by  a  machine  tool  that  is  wearing  out. 

A  special  source:  an  unlikely  event  such  as  the  breaking  of  an 
artifact  being  processed  by  machine  tool  that  fails 
unexpectedly 
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Abstract 

In  the  real  world,  “Money  Allocated  is  Money  Spent"  (MAIMS).  As  a  consequence,  cost 
underruns  are  rarely  available  to  protect  against  cost  overruns,  while  task  overruns  are  passed 
on  to  the  total  project  cost.  The  combination  of  the  probabilistic  aspects  of  project  costs  and  the 
MAIMS  principle  have  important  implications  for  budget  allocation  and  the  management  of 
contingencies.  Project  costs  depend  not  only  on  the  desired  probability  of  success  but  also  on 
budget  allocation  and  contingency  management.  This  is  in  contrast  with  both  deterministic 
practices  that  allocate  a  percentage  of  the  project  baseline  cost  for  contingency  as  well  as 
today's  de-facto  probabilistic  cost  analyses  that  provide  a  cost  contingency  independent  of  the 
budget-allocation  strategy.  The  realistic  modeling  of  cost  uncertainties  and  the  MAIMS  principle 
provide  a  framework  for  developing  a  viable  cost-management  strategy  for  allocating  baseline 
budgets  and  contingencies.  Based  on  this  analysis,  the  project  manager  can  maintain  a 
realistic  project-wide  contingency  and  dynamically  distribute  it  to  the  individual  risks  on  an  as- 
needed  basis.  Projects  that  implement  dynamic  cost-contingency  management  based  on  these 
principles  are  likely  to  achieve  a  higher  probability  of  success  and  cost  less. 

Introduction 

Real-world  experience  and  intuition  both  suggest  that  project  costs  depend  on  many 
factors,  including  technical,  organizational,  and  behavioral  considerations.  Thucydides  got  to 
the  very  root  of  the  cost-overrun  problem  over  2000  years  ago  when  he  stated,  “Their  judgment 
was  based  more  on  wishful  thinking  than  on  sound  calculation  of  probabilities”  (Augustine, 

1997,  p.255). 

In  the  1990's,  the  Lockheed  Missiles  and  Space  Co.  carried  out  a  study  which  concluded 
that  the  following  deficiencies  in  cost  modeling  and  contingency  management  have  been  major 
contributors  to  both  project  high  costs  and  overruns  (Gordon,  1997): 
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■  Hidden  incentives  in  procurement 

■  Hidden  incentives  in  management  styles 

■  Failure  to  coordinate  cost  analysis  and  cost  management 

■  Use  of  invalid  mathematics  such  as  arithmetically  summing  uncertain  cost  elements 

instead  of  using  statistical  methods 

■  Overlooking  the  "Money  Allocated  Is  Money  Spent"  (MAIMS)  principle 

The  MAIMS  principle  accounts  for  the  fact  that  projects  rarely  underrun  their  allocated 
budgets.  It  is  the  money  analog  of  Parkinson’s  Law,  “Work  expands  to  fill  the  time  allotted.” 

The  principle  is  also  in  concordance  with  Goldratt’s  observation  that  negative  human  behavior  is 
a  major  cause  of  the  project-scheduling  problem.  Goldratt  (1997)  developed  the  Critical  Chain 
Project  Management  (CCPM)  as  a  management  philosophy  and  solution  that  simultaneously 
reduces  project  duration  and  protects  against  schedule  risk.  A  key  principle  of  CCPM  is  to 

aggregate  task  buffers  at  the  project-level  for  use  where  and  when  needed.  But  it  also 

proposed  the  following  guidelines  for  sizing  buffers:  (1)  cut  task  duration  estimates  in  half,  and 
(2)  add  approximately  25%  of  the  original  estimate  to  the  project  buffer.  These  guidelines 
appear  to  be  rather  arbitrary,  and  many  technical  managers  are  uncomfortable  with  them.  A 
number  of  simple  alternatives  to  estimate  and  sum  buffers  have  been  proposed  (Newbold, 

1 998;  Schuyler,  2001 ).  We  think  that  their  use  is  no  longer  justified  because  of  the  availability  of 
simple  Monte  Carlo  simulation  tools  such  as  @Risk®  and  Crystal  Ball®. 

The  premise  of  this  paper  is  that  a  credible  Probabilistic  Cost  Analysis  (PCA)  needs  to 
integrate  findings  on  human  behavior  with  mathematically  valid  models  and  sound  management 
techniques  to  obtain  realistic  cost  estimates  and  achieve  project  success.  A  key 
recommendation  is  that  in  order  to  deliver  successful  projects  at  an  optimal  cost  project, 
management  needs  to  allocate  "reasonable"  budgets  to  the  cost-account  managers  and 
dynamically  manage  the  cost-contingency  funds  as  a  risk  portfolio  at  the  program/project  level. 

Proposed  Modifications  to  Today’s  Typical  PCA 

Assessing  Uncertain  Cost  Elements 

R&D  and  complex  engineering  projects  rely  heavily  on  engineering/expert  judgment  for 
the  assessment  of  uncertain  cost  elements.  Unfortunately,  these  subjective  assessments  are 
often  performed  in  a  rather  ad-hoc  manner,  and  they  have  been  identified  as  a  critical  source  of 
error  in  probabilistic  risk  analyses  (Keeney  &  von  Winterfeld,  1991).  The  Direct  Fractile 
Assessment  (DFA)  method  has  been  investigated  in  numerous  psychological  experiments  and 
found  to  provide  one  of  the  most  reliable  and  least  bias-prone  procedures  for  eliciting  uncertain 
quantities  (Alpert  &  Raiffa,  1982).  We  recommend  that  experienced  analysts  and  domain 
experts  determine  the  10th,  50th,  and  90th  percentiles  for  uncertain  cost  elements.  While  other 
percentiles  may  be  used,  these  seem  to  be  highly  practical  (Dillon,  John,  &  von  Winterfeld, 
2002). 


Fitting  Cost  Elements  with  Realistic  Probability  Density  Functions  (PDF) 

Uncertain  cost  elements  are  more  appropriately  modeled  as  continuous  than  discrete 
random  variables.  We  favor  the  use  of  the  three-parameter  Weibull  distribution  because  it  is  an 
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open-ended  function  that  can  assume  a  wide  variety  of  shapes  (Kujawski,  Alvaro,  &  Edwards, 
2004).  It  is  also  more  flexible  than  the  three-parameter  lognormal  even  though  both  are 
characterized  by  three  independent  parameters.  The  use  of  more  complex  PDFs  seemed 
unwarranted  for  fitting  three  subjectively  assessed  percentiles.  Analysts  and  assessors  should 
always  validate  that  they  feel  comfortable  with  the  shape  of  the  fitted  distribution. 

Incorporating  the  MAIMS  Principle 

The  MAIMS  principle  plays  a  significant  role  in  PCA.  Once  a  cost  element  is  allocated  a 
budget,  x*  it  becomes  a  random  variable  with  minimum  value  x*  rather  than  the  lower  range  of 
the  original  PDF.  The  cost  element  is  then  given  by  a  PDF  with  a  delta-like  function31  (or  spike) 
at  x*  that  accounts  for  all  random  values  less  than  or  equal  to  x*  and  the  original  distribution  for 
values  greater  than  x*.  The  associated  Complementary  Distribution  Function  (CDF)  has  a  step- 
function  behavior  at  x*  and  is  identical  to  the  original  CDF  above  x*.  The  effect  on  the  cost 
element  is  that  its  mean  increases  and  its  standard  deviation  decreases  with  increasing  values 
of  x*.  As  a  result,  the  MAIMS  principle  plays  a  significant  role  in  budget  management. 

Modeling  Specific  Risks 

The  above  PDFs  provide  a  macroscopic  rather  than  a  microscopic  view  of  the  project 
cost  risk.  They  effectively  model  those  factors  or  project  characteristics  that  are  ever  present 
and  contribute  to  cost  uncertainties.  But  complex  projects  often  involve  a  number  of  critical 
decisions  and  high-impact  risks  which  call  for  explicit  risk-mitigation  actions.  A  detailed  PCA 
should  incorporate  both  the  macroscopic  and  microscopic  views  to  ensure  that  all  risks  and  cost 
uncertainties  are  addressed  and  that  the  risk-reduction  activities  are  transparent  (Chapman  & 
Ward,  1997).  The  analysis  of  specific  risks  and  risk-response  actions  requires  a  microscopic 
view  and  is  best  carried  out  using  tools  such  as  decision  trees,  influence  diagrams,  or  other 
discrete  representations  (Kujawski,  2002a). 

Modeling  Correlations 

Cost  elements  are  correlated  because  project  characteristics  (such  as  complexity, 
criticality,  management,  staff,  and  processes)  are  likely  to  impact  multiple  cost  elements  at  the 
subsystem  and  system  levels.  Also,  the  realization  of  any  one  risk  is  likely  to  influence  other 
risks  and  to  increase  their  probabilities  and/or  consequences.  Kujawski,  Alvaro,  and  Edwards 
(2004)  have  developed  a  Two-Level  Correlation  Model  (TLCM)  which  greatly  reduces  the 
number  of  parameters  needed  to  specify  a  mathematically  valid  and  physically  realistic 
correlation  matrix.  In  its  simplest  form,  it  models  correlations  among  cost  elements  of  the  same 
and  different  subsystems  with  only  two  parameters,  pint  and  pext- 

Application  to  a  Representative  Design  and  Engineering  Project 

To  investigate  the  concepts  and  issues  discussed  in  the  previous  sections,  we  consider 
a  hypothetical  project  with  three  level-2  cost  elements  (project/system-level  and  two 


31  Caution:  The  MAI  MS-modified  PDFs  are  not  the  same  as  the  Crystal  Ball®  and  @Risk® 
truncated  PDFs. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  554- 


subsystems)  each  with  three  level-3  cost  elements.  Figure  1  depicts  different  budget-allocation 
strategies  for  a  given  set  of  PDFs  and  TLCM  parameters32. 


100%  7 


6,000  6,500  7,000  7,500  8,000  8,500  9,000  9,500 


Cost,  K$ 


— * — MAIMS  @  X50 

*  MAIMS  @  mean 

- MAIMS  &,  X75 

Ideal 

Figure  1.  Illustrative  Impact  of  Different  Budget  Allocation  Strategies  on  Project  Cost 

Note:  The  cost  elements  are  modeled  with  Weibull  distributions  fitted  to  the  10th,  50th,  and  90th 
fractiles.  The  cost  correlations  are  modeled  using  the  two-level  correlation  model  with  parameters  of  0.6 
and  0.4. 


The  “ideal  curve”  corresponds  to  the  model  where  the  project  staff  rationally  spends 
money  only  as  necessary  to  satisfy  the  project  requirements.  In  this  ideal  world,  the  actual 
costs  may  be  less  than  the  budgeted  costs,  and  the  savings  are  available  to  support  other 
project  elements  on  an  as-needed  basis.  In  the  MAIMS_@_X50  and  MAIMS_@_X75  curves, 
all  cost  elements  are  allocated  equal  percentiles  of  50%  and  75%,  respectively.  The 
MAIMS_@_mean  curve  corresponds  to  the  case  in  which  each  cost  element  is  allocated  its 
mean  or  expected  value.  Each  cost  element  is  then  budgeted  at  a  percentile  that  depends  on 
the  shape  of  the  assessed  PDF.  The  MAIMS  effects  increase  with  higher  allocated  budgets  and 
are  substantial  over  a  wide  range  of  Probability  of  Success  (PoS)  values  of  interest  to  PCA. 

Budget  Allocation,  Contingency,  and  Project  Cost 

Our  objective  is  to  integrate  the  presented  concepts  into  a  sound  methodology  for 
determining  an  optimal  as  well  as  realistic  Total  Estimated  Cost  (TEC)  and  budget- 
allocation/management  strategy  for  a  given  PoS.  The  combination  of  cost  uncertainties  and  the 
MAIMS  principle  complicates  the  situation.  As  we  have  shown,  the  TEC  depends  not  only  on 


32  The  calculations  were  performed  using  Crystal  Ball®  and  10,000  trials. 
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the  desired  PoS  but  also  the  budget  allocation  and  the  management  of  contingencies.  The 
project  cost  cannot  be  estimated  until  the  cost  management  strategy — including  budget 
allocation — is  specified.  We  like  to  think  that  this  contains  a  flavor  of  the  Heisenberg 
Uncertainty  Principle. 

Much  has  been  written  on  cost  contingency;  but  there  is  still  much  confusion  (Baccarini, 
1999;  INCOSE,  2003).  To  shed  additional  light  on  the  subject,  we  express  the  Management 
Cost  Contingency  (MCC)  in  a  form  that  exhibits  its  dependence  on  the  PoS  and  the  cost 
management  strategy: 

MCC(PoS,  PBCi,...,  PBCn)  =  TEC(PoS,  PBCi,...,  PBCn)-PBC. 

PBC,  is  the  baseline  budget  for  cost  element  Ci;  and  PBC  is  the  probabilistic  sum  of  all 
the  project  cost  elements.  The  above  relationship  contrasts  with  both  (1)  the  deterministic 
practice  that  allocates  a  percentage  of  the  PBC  as  MCC,  and  (2)  today's  typical  PCA  that 
provides  a  MCC  that  is  independent  of  the  budget-allocation  strategy.  Figure  2  depicts  the 
TECs  and  MCCs  corresponding  to  Figure  1. 


Confidence  Level  of  Total  Estimated  Cost 


. TEC  MAIMS  @  mean 

- TEC  MAIMS  @_75 

TEC  MAIMS_@_50 

—  o —  TEC  Ideal 

-  -  -  A-  -  -  MCC  MAIMS  @  mean 

—A —  MCC  MAIMS_@_75 

—  -  A—  -  MCC  MAIMS_@_50 

Figure  2.  Impact  of  Different  Cost  Management  Strategies  on  the  Cost  and  Contingency 

for  the  Project  in  Figure  1 

Figure  2  contains  valuable  information  for  both  the  procuring  activity  and  the  contractor. 
The  budget  management  strategy  has  a  significant  impact  on  the  TEC  for  a  given  PoS.  The 
effects  of  the  MAIMS  principle  increase  with  increasing  budget  allocations  and  are  substantial 
for  all  but  the  very  highest  PoS  values.  The  MAIMS  principle  has  little  impact  at  the  very  high 
confidence  levels  (CL  >  95%)  because  each  contributing  cost  element  must  then  be  near  its 
maximum  or  100th  percentile  value.  These  results  have  important  implications  for  cost 
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management.  For  example,  sizeable  cost  reductions  are  achieved  by  allocating  budgets  to  the 
cost  elements  at  the  50%  CL  rather  than  the  75%  CL.  The  standard  PCA  that  assumes  an 
“ideal”  project  provides  a  false  sense  of  confidence;  it  may  be  a  major  source  of  cost  overruns 
even  for  projects  with  high  contingencies. 

Why  Projects  Even  with  High  Cost-Contingencies  Often  Fail 

Consider  a  hypothetical  request  for  proposal  for  the  project33  depicted  in  Table  1 .  To 
level  the  playing  field,  the  procuring  activity  specifies  that  all  bids  should  provide  the  50%  CL 
cost.  Contractor  A  has  a  certain  level  of  sophistication.  He34  prepares  a  typical  PCA  with  every 
bid;  but  he  is  not  cognizant  of  the  MAIMS  principle.  He  performs  today's  typical  PCA  and 
obtains  the  CDF  in  Fig.  2  labeled  “TEC  Ideal”  and  a  P50  TEC  of  7,348  K$.  Based  on  this 
analysis,  Contractor  A  submits  a  bid  of  7,348  K$  and  rationalizes  that  the  proposal  is 
conservative  given  that  the  P50  value  is  30%  above  the  low  estimate  of  5,633  K$.  But  because 
of  the  MAIMS  principle,  Contractor  A's  risks  are  significantly  greater  than  he  thinks.  Once  the 
contract  is  awarded,  management  proceeds  to  baseline  and  allocate  budgets  to  the  cost 
elements  at  their  mean  values.  Given  that  the  cost  elements  are  budgeted  at  their  mean 
values,  the  TEC  is  really  given  by  the  CDF  in  Fig.  2  labeled  “PEC  MAIMS_@_mean,”  the  P50 
TEC  is  8,071  K$,  and  the  PBC  of  7,665  K$  is  the  lowest  achievable  cost.  To  management’s 
surprise,  this  value  is  317  K$  less  than  the  proposal  bid  of  7,348  K$.  Because  of  the  MAIMS 
principle,  there  is  a  negligible  likelihood  that  Contractor  A,  given  his  practices,  can  deliver  the 
project  for  the  submitted  bid  of  7,348  K$.  Table  1  summarizes  this  and  several  other  scenarios. 
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7,348 

-1,118 

-13% 

0% 

80% 

9,330 

864 

10% 
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8,626 

160 

2% 

52% 

Table  1.  Some  Summary  Data  for  the  Different  Cost  Management  Strategies  Depicted  in 

Figure  2 


Similarly,  when  considering  specific  technical  risks,  the  common-sense  and 
mathematically  valid  solution  for  efficient-cost  risk  management  is  to  maintain  a  project-wide 
contingency  and  to  distribute  it  to  the  individual  risks  on  an  as-needed  basis.  This  approach  to 


33  This  is  the  same  illustrative  project  used  for  Figures  1  and  2. 

34  Gender  neutral 
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managing  project  technical  risks  may  be  thought  of  as  a  variant  of  modern  portfolio  theory 
(Markowitz,  1991).  The  implication  to  managing  project  is  that  less  attention  should  be  given  to 
the  individual  risks  (substituted  for  stocks)  and  more  to  the  project  (substituted  for  portfolio)  as  a 
whole  (Kujawski,  2002b).  CCPM  formalizes  analogous  concepts  and  their  implementation  for 
project  schedule  planning  and  management. 

Concluding  Remarks 

This  paper  develops  a  practical  and  sound  framework  for  quantifying  the  influence  of 
human  behavior  on  project  cost  and  efficiently  managing  project  risks  and  cost  contingencies. 
The  key  elements  include: 

■  The  incorporation  of  the  “Money  Allocated  Is  Money  Spent”  (MAIMS)  principle.  The 
probability  distribution  of  each  cost  element  is  modified  by  setting  all  cost  values  less 
than  the  allocated  budget  to  the  allocated  budget  in  the  MCS. 

■  The  realistic  assessment  of  cost  uncertainties  and  technical  risks  using  proven  methods 
such  as  the  Direct  Fractile  Assessment  method,  event  trees  and/or  influence  diagrams. 

■  The  realistic  treatment  of  correlations  among  cost  elements. 

■  The  probabilistic  treatment  of  the  cost  elements  and  explicit  representation  of  technical 
risks.  The  analysis  is  readily  performed  using  commercially  available  Monte  Carlo 
simulation  Excel  add-ins. 

■  The  implementation  of  a  project-wide  cost  contingency  to  ensure  the  contractually 
agreed-to  or  acceptable  probability  of  success. 

■  Contingencies  held  and  managed  at  the  project-wide  level.  They  should  not  be 
allocated  at  the  task-level  and  held  by  individual  subsystem  managers. 

■  A  dynamic  allocation  algorithm  with  system-level  oversight  for  managing  project  risks. 
However,  individual  technical  risks  are  still  managed  by  the  responsible  technical 
performers.  A  database  for  tracking  risks  would  provide  a  powerful  tool  for  accurately 
watching  and  forecasting  contingency  allocations  and  for  controlling  adverse  behaviors. 

All  seven  principles  are  necessary  to  ensure  that  adequate  contingencies  are  available 
to  mitigate  all  project  risks  and  not  just  selected  ones.  Projects  that  implement  all  seven 
principles  are  more  likely  to  succeed  and  cost  less. 

The  author  acknowledges  that  it  takes  effort  to  develop  these  more  realistic  models  and 
that  all  models  are  only  approximations  to  reality.  The  single  greatest  challenge  to  the 
development  and  use  of  improved  probabilistic  cost  analysis  and  dynamic  budget  allocation  is 
the  implementation  of  systems  thinking  at  the  personnel,  organizational,  and  institutional  levels. 
But  given  the  magnitude  of  the  cost  overrun  problem,  there  is  no  excuse  for  accepting  the  status 
quo;  the  benefits  are  likely  to  be  significant. 
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Abstract 

Since  the  end  of  the  Cold  War,  the  perceived  need  for  Navy  ships  has  dropped,  and  so 
the  shipbuilding  budget  has  dropped.  Seemingly  coincidental  with  this  budgetary  pressure,  and 
perversely  aggravating  the  problem,  ship  costs  began  to  rise  steeply.  We  will  set  aside  that 
ships  have  grown  in  weight  by  about  three  percent  per  year  since  World  War  II  and  that  ever¬ 
more  weapon  systems  are  being  put  into  them,  and  confine  ourselves  to  discussions  of  costs 
rising  for  ships  beyond  the  increase  in  “content.”  We  will  also  set  aside  rises  due  to  commodity 
prices  and  inflation  and  that  fewer  ships,  divided  among  a  fixed  industrial  base,  reduce  the  base 
for  overhead  and  reduce  opportunity  for  the  effects  of  quantity-driven  learning;  these  effects  are 
well  understood,  and  yet  cost  growth  in  ships  exceeds  that  which  they  explain.  This  paper  will 
show  two  additional  effects,  each  of  which  causes  ship  direct  labor  to  rise  in  a  way  that  has 
never  been  adequately  modeled.  The  paper  will  demonstrate,  via  a  statistically  significant 
model,  cost  growth  both  from  loss  of  learning  due  to  increased  time  between  ship  starts  as  well 
as  from  the  lessening  of  efficiency  due  to  inexperienced  labor  caused  by  fluctuating  demand. 

Introduction 

Since  the  end  of  the  Cold  War,  the  perceived  need  for  Navy  ships  has  dropped; 
therefore,  the  shipbuilding  budget  has  dropped,  as  has  most  of  the  Defense  Department’s 
budget.  The  Global  War  on  Terror  (GWOT)  has  further  stretched  budgets.  Seemingly 
coincidental  with  this  budgetary  pressure,  and  perversely  aggravating  the  problem,  ship  costs 
began  to  rise  steeply.  This  paper  will  demonstrate  that  a  significant  portion  of  the  increase  in 
ship  costs  is  not  a  coincidence  at  all  but  is  a  direct  result  of  decreased  budgets  in  an 
unavoidable  way.  We  will  set  aside  that  ships  have  grown  in  weight  by  about  three  percent  per 
year  since  World  War  II  and  that  ever-more  weapon  systems  are  being  put  into  them,  and 
confine  ourselves  to  discussions  of  costs  rising  for  ships  beyond  the  increase  in  “content.” 
Conventional  wisdom  holds  that  fewer  ships,  divided  among  a  fixed  industrial  base,  causes 
reduced  base  for  overhead  and  reduced  opportunity  for  the  effects  of  quantity-driven  learning; 
these  effects  are  fairly  well  understood,  and  yet  cost  growth  in  ships  exceeds  the  cost  growth 
explained  by  conventional  effects.  This  paper  will  show  that  two  additional  effects  are  at  play, 
each  of  which  causes  individual  ship  direct  labor  to  rise  in  a  way  that  existing  models  and  cost 
estimating  tools  do  not  predict.  The  paper  will  explain  these  two  effects  and  conclusively  show 
their  impact. 

The  analysis  that  follows  will  demonstrate,  via  a  statistically  significant  model,  two 
effects:  the  effect  of  less  demand  and  the  effect  of  less-steady  demand.  The  former  causes 
stretched-out  ship-class  acquisitions,  increasing  time  between  ship  starts;  the  latter  causes 
workforce  instability.  The  analysis  was  conducted  first  on  the  Arleigh  Burke  (DDG  51)  Aegis- 
guided  missile  destroyer  program  and  demonstrated  to  be  statistically  significant  (25  data 
points).  In  order  to  be  absolutely  certain  of  the  analysis,  as  is  the  custom  on  important  studies, 
it  was  conducted  a  second,  independent  time,  on  a  wholly  different  ship  type,  the  Wasp  (LHD  1) 
class  of  amphibious  assault  ships  (7  data  points).  These  ships  are  nearly  four  times  as  big  as  a 
DDG,  have  a  different  mission,  and  were  built  in  the  same  era,  but  on  a  significantly  different 
schedule.  The  analysis  revealed  the  same  result  and  was  again  statistically  significant.  The 
import  of  this  second  trial  cannot  be  overstated;  it  results  in  dramatic  increase  in  the  significance 
of  the  results,  from  the  customary  95%  significance  level  to  a  99.75%  level.  The  resultant 
model  is  called  the  Advanced  Learning  Model.  (The  three  progressively  more  comprehensive 
versions,  ALM  1 ,  ALM  2,  and  ALM  3,  are  summarized  in  this  paper.) 
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Graphics  would  greatly  simplify  and  clarify  this  summary,  but  the  necessity  of  removing 
values  and  avoiding  the  compromise  of  proprietary  data  are  delaying  this  process.  The  paper 
as  presented  will  show  these  very  illuminating  graphics.  The  authors  apologize  for  the  lack  of 
graphics  herein  and  hope  the  reader  will  attend  the  briefing! 

ALM  1  Analysis:  The  DDG  Class 

The  analysis  begins  with  an  examination  of  Vessel  Labor  (VL),  recurring  direct  labor,  in 
manhours  (MH),  by  ship.  The  problem  that  presents  itself,  in  ship  class  after  ship  class,  is  that 
there  is  a  fairly  obvious  learning  curve  for  the  beginning  of  the  ship  class,  but  at  some  point  this 
curve  fails  to  predict  future  ships,  and  VL  begins  to  rise  seemingly  without  cause.  If  we  simply 
run  a  learning  curve  through  the  initial  ships,  and  then  run  subsequent  curves  as  the  number  of 
ships  rises,  we  will  get  an  answer  that  seems  to  show  an  ever-flattening  learning  curve.  (As  we 
note  later,  some  parties  even  claim  that  learning  ceases  at  some  point  in  a  ship  class;  this  is  a 
misinterpretation  of  the  perturbing — and,  as  we  will  show,  perturbed — data.) 

This  is  a  learning  curve  only  in  a  manner  of  speaking,  because,  at  some  point,  the 
“learning  curve”  does  not  pass  through  the  data  points  in  any  satisfactory  way.  There  is  no 
“learning  curve”  other  than  in  the  sense  that  any  set  of  points  can  have  a  best-fit  line  run 
through  them.  What  is  going  on  here? 

One  of  the  problems  is  that  traditional  learning  curve  theory  requires  a  steady  work  force 
building  the  same  product  multiple  times  with  no  significant  interruptions  or  pauses.  Modern 
ship  programs  aren’t  like  that.  If  these  conditions  are  not  fulfilled,  there  is  discussion  in  the 
literature  of  loss  of  learning,  but  no  closed-form,  statistically-based  method  to  predict  how  much 
learning  is  lost.  One  model,  the  Anderlohr  Break-in-Production  Model,  quantifies  the  effects  of 
production  breaks,  but  requires  expert  opinion  and  is,  therefore,  not  defensible — it  is  only  useful 
when  mutually  agreed  to  by,  e.g.,  the  contractor  and  the  contracting  officer.  In  practice,  this 
agreement  is  rare.  In  competition,  or  in  cases  of  outside  scrutiny,  for  example  by  the  OSD 
CAIG,  this  method  will  almost  certainly  not  stand  up. 

This  problem  seemed  intractable,  with  no  defensible  methodology,  until  one  graph 
changed  everything;  that  single  graph,  one  of  many  scatter  plots,  led  to  the  breakthrough  that 
started  the  ALM.  In  that  graph,  Vessel  Labor  and  Time  between  Ship  Deliveries  (in  the  final 
version,  start  fabrication  was  used  as  the  marker  for  ship  construction  interval)  were  jointly 
plotted  by  ship  (this  necessitated  use  of  a  dual-axis  graph).  From  this  one  “Rosetta  Stone” 
graphic,  it  became  clear  that  MH  rise  and  fall  as  time  between  ship  starts  rises  and  falls.  We 
were  not  yet  ready  to  assert  a  closed-form  learning  model,  but  we  began  to  suspect  that 
learning  was  continuing  throughout  the  ship  class,  and  perhaps  being  lost  as  well.  The  driving 
variable  was  ultimately  expressed  in  terms  of  percent  of  build  duration  instead  of  days,  as  will 
be  explained  later.  Many  painstaking  analytic  steps  remained,  but  the  basic  answer  had  begun 
to  emerge. 

The  analysis  took  a  number  of  steps,  as  we  have  said.  It  should  be  noted  that  nothing 
was  easy  in  this  analysis,  since  the  data  went  back  many  years  and,  more  often  than  not,  was 
not  available  in  the  form  in  which  it  was  needed.  Accordingly,  practically  every  step  required 
data  normalization  using  analytical  constructs  or  some  form  of  “decoding”  or  deconstructing. 
These  steps  were  all  done  after  careful  consultation  with  people  who  were  present  and  in 
responsible  positions  at  the  time  of  production,  and  are  in  accord  with  conventional  wisdom. 
Most  significantly,  in  the  end  the  analysis  works,  so  the  steps  are,  to  a  point,  justified.  We  say 
this  knowing  that,  “the  end  does  not  justify  the  means”;  nevertheless,  when  steps  that  make 
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sense  are  taken,  and  the  result  is  both  in  accord  with  our  ideas  as  to  how  it  should  be,  and 
statistically  verifiable,  the  result  acts  to  shore  up  our  confidence  in  the  steps.  With  that  warning, 
we  will  proceed  with  the  description  of  the  analysis. 

1 .  First,  we  subtracted  Type  I  change  orders  as  listed  in  the  contracts  for  each  ship.  These 
are  changes  requested  by  the  Navy  for  new  “content”  on  the  ship. 

2.  Second,  we  removed  change  orders  that  were  unseen  originally  because  they  were 
embedded  in  the  base  work  of  later  ships.  These  change  orders  were  no  longer  tracked 
in  the  tracking  system,  as  the  work  they  contained  was  “rolled  into”  the  base  contract. 

3.  Third,  we  took  into  account  “Green  Labor,”  adjusting  the  MH  to  what  they  would  have 
been  had  the  labor  mix  been  a  nominal  mix  of  experienced  and  inexperienced  labor. 

This  third  step  removes  ship-by-ship  variability  of  the  labor  force  mix. 

4.  Fourth,  we  adjusted  for  time  between  ship  starts,  as  represented  by  the  Start  Fabrication 
step.  This  fourth  step  is  the  keystone  to  the  analysis.  The  statistics  for  this  step  will  be 
shown. 

The  first  three  steps  were  difficult  to  judge  on  a  one-at-a-time  basis  since  they  were  not 
statistically  testable.  As  we  have  said,  however,  we  assert  that  the  final  step,  having  been 
performed  after  the  first  three  and  tested  as  valid,  acts  to  indicate  (and  we  use  this  word 
advisedly)  that  the  first  three  steps  were  valid.  Had  the  fourth  step  failed,  we  would  not  know 
which  step(s)  to  blame,  but  since  it  worked,  and  since  all  four  steps  conform  with  conventional 
wisdom  on  how  these  effects  should  work,  we  feel  justified  in  claiming  they  all  were  valid.  We 
did  not  rest  on  this  claim,  however,  but  conducted  analysis  on  a  second  ship  class  and 
ascertained  that  our  findings  were  neither  a  coincidence  nor  a  result  of  “wishful  analysis.” 

These  steps  as  described  result  in  a  “peeled  onion” — by  “backing  out”  the  various 
compounding  effects  to  arrive  at  what  turns  out  to  be  the  smooth  and  continuous  underlying 
learning  curve.  It  should  be  noted  that  all  steps  can  be  reversed  to  constitute  a  cost-estimation 
model  that  relies  on  accurate  plans  for  change  orders,  yard  manning,  and  ship  intervals,  at  least 
the  last  two  of  which  can  be  derived  from  shipbuilding  plans. 

Analytical  Details 

Analysis  for  Step  1 — Change  Orders 

Change  orders  were  available  for  all  ships,  but  only  at  the  aggregate  level.  When  these 
aggregated  changes  were  removed,  it  was  clear  further  analysis  was  needed.  We  knew  from 
testimony  that,  at  a  certain  point  in  the  life  of  any  class,  change  orders  from  prior  ship  contracts 
are  “rolled  into”  the  base  contract;  from  this  point  forward,  these  change  orders  are  not  tracked 
separately  from  the  base  work.  This  led  us  directly  to  Step  2,  embedded  change  orders. 

Before  we  depart  from  step  1 ,  however,  we  should  note  that  actual  VL  MH  for  change 
orders  are  not  tracked  separately;  so,  only  estimated  change  orders  are  available.  There  is 
considerable  doubt  among  analysts  as  to  the  value  of  change  orders,  some  believing  that 
change  orders  are  significantly  overestimated  (and,  hence,  a  great  source  of  revenue),  others 
believing  they  are  a  significantly  underestimated  (and,  hence,  a  loss).  When  this  analysis  was 
done,  we  emerged  with  the  conviction  that  change  orders  are  estimated  quite  well,  else  the 
analysis  would  never  have  held  up;  we  urge  analysts  and  decision-makers  to  consider  change 
order  estimates  to  be  more  accurate  than  they  have  in  the  past. 
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Analysis  for  Step  2 — Embedded  Change  Orders 

The  removal  of  embedded  changes  required  a  great  deal  of  decryption.  Using  the 
aggregate  changes  as  a  base,  we  segregated  out  one-time  allowances  for  rip-out  (removing 
completed  or  in-progress  work  that  the  change  order  replaces),  disruption,  and  one-time  change 
orders.  We  also  had  to  apply  learning  to  the  change  orders,  because  like  any  work,  as  units 
progress,  MH  decrease.  Applying  the  normal  percentage  for  each  of  these  one-time  effects, 
and  applying  learning,  we  “reverse  engineered”  to  decode  the  change  orders  for  follow-on  ships. 

With  the  ship-by-ship  deconstructed  change  orders  in  hand,  we  were  able  to  “see”  what 
the  change  orders  looked  like  and,  thus,  could  see  when  the  change  orders  were  rolled  into  the 
basic  contract.  We  were  able  to  spot  a  distinct  rise  in  base  work  and  an  offsetting  drop  in 
change  orders  on  the  same  ship;  we  matched  this  detective  work  with  the  memories  of  senior 
shipyard  personnel  and  with  the  contractual  picture  (changes  in  the  base  work  could  only 
happen  at  the  start  of  a  contract  of  multiple  ships).  The  resulting  embedded  change  orders 
were  then  subtracted  from  the  appropriate  ships;  total  changes  were  now  removed. 

Analysis  for  Step  3 — Green  Labor 

“Green  Labor”  is  defined  as  workers  with  less  than  5  years  experience;  after  5  years, 
Green  Labor  becomes  “Dry”  or  “Seasoned”  Labor.  Green  Labor  is  less  effective  than  Seasoned 
Labor,  based  upon  a  study  conducted  at  Northrop  Grumman  Ship  Systems  at  their  Ingalls 
Operations,  and  as  indicated  by  a  percentage.  (For  example,  if  a  worker  is  80%  effective,  then 
it  will  take  him  or  her  25%  more  hours  to  complete  a  task,  since  1 .0/0.8  =  1 .25.)  This  old  study 
was  not  much  in  use,  because  it  was  perceived  to  be  less  than  defensible  since  it  was  based 
upon  expert  (inside-yard)  opinion.  One  of  the  benefits  of  our  analysis  was  that  we  revitalized 
this  model  and  gave  it  credibility,  as  we  will  discuss.  The  percentages  by  year  probably  differ 
with  type  of  ship,  yard,  training  programs,  etc.,  as  we  know  that  the  percentages  are  different  at 
Northrop  Grumman  Newport  News,  based  upon  a  study  conducted  there.  For  this  analysis,  we 
used  a  weighted  average  of  effectiveness  for  workers  in  the  first  5  years. 

As  in  earlier  steps,  however,  it  wasn’t  just  that  easy.  Data  did  not  exist,  for  any  unit, 
tracking  the  portion  of  the  labor  force  that  was  green  vs.  the  portion  of  the  labor  force  that  was 
dry.  Again,  we  needed  a  construct.  By  interviewing  managers  and  HR  personnel,  we 
ascertained  the  key  variables  in  the  experience-level  problem,  and  built  a  Markov  Chain  model 
for  labor  experience.  We  know,  for  example,  that  layoffs  were  LIFO  (last-in-first-out);  we  know 
the  attrition  rates  for  Seasoned  and  Green  labor,  and  we  know  that  most  hired  labor  was  Green. 
We  built  a  Markov  Chain  model,  and  it  produced  the  then-current  labor  mix  within  about  2 
percentage  points;  the  model  was  seeded  with  a  reasonable  start  point,  so  we  were  confident 
that  the  Markov  Chain  labor  mix  estimates  were  close.  This  model  gave  us  the  year-by-year 
labor  mix,  and  we  used  this  mix  to  adjust  to  a  notional  (most  common)  labor  mix,  thereby  nearly 
(because  this  is  an  approximation,  we  can  only  say  “nearly”)  eliminating  the  effects  of  labor 
force  mix. 

As  a  side  note,  in  past  briefings  we  often  said  that  the  effects  of  labor  mix  are  not 
dominant  in  this  problem,  because  labor  mix  is  fairly  stable;  after  Hurricane  Katrina,  however, 
when  we  looked  at  reconstituting  the  labor  force  with  this  model,  we  saw  that  when  labor  swings 
considerably,  the  effect  becomes  quite  big.  In  a  small-business-base  case,  considerable  cost 
impact  is,  in  fact,  caused  by  labor  swings.  While  average  Green  Labor  is  less  efficient  than 
Seasoned  Labor,  first-year  labor  is  quite  a  bit  less  efficient,  so  an  unstable  work  force  that  is 
constantly  ramping  up  and  down  will  result  in  considerable  loss  of  efficiency.  This  all  said, 
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however,  our  model  demonstrates  that  the  potential  for  cost  impacts  are  greatest  in  the  last 
variable:  loss  of  learning  due  to  interval  effect. 

Analysis  for  Step  4 — Loss  of  Learning  due  to  Interval  Effect 

The  final  step  is  the  most  important  step,  both  because  the  prior  steps  were  either 
understood  or  in  practice,  and  because  this  step  acts  to  verify,  in  the  way  we  have  described, 
the  earlier  steps.  In  this  step,  we  conducted  a  linear  regression  of  the  difference  between 
learning  as  observed  and  learning  as  expected.  Though  this  was  difficult  in  derivation,  and  had 
many  false  steps,  the  result  is  intuitive. 

Two  major  assumptions  underlie  our  method.  These  two  assumptions  are  in  accord  with 
current  learning  curve  theory  but  deserve  mention.  First,  we  assumed  that  learning  is  constant 
and  incremental  and  proceeds  from  one  ship  to  the  next.  This  means  that  as  other  effects 
occur,  the  learning  curve  factor  is  still  at  play.  We  have  been  surprised  that  others  have 
asserted  that  learning  stops  after  nine  or  ten  ships35.  This  is  not  in  consonance  with  anything  in 
the  literature  of  learning,  and  we  believe  that  the  results  of  this  analysis  bear  out  our  belief  that 
learning  continues  through  all  units.  Getting  ahead  of  ourselves  for  a  moment,  we  believe  that 
at  some  point  in  a  ship  class  there  is  surely  a  considerable  change-order  effect  and  the 
attendant  slow-down  in  production  as  the  changes  are  engineered,  which  causes  loss  of 
learning.  This  is  not  to  say  learning  stopped,  but  rather  to  say  that  other  effects  are  causing  a 
rise  in  ship  costs.  Some  of  these  effects  are  controllable;  and  all  are,  to  some  degree, 
predictable.  Second,  we  believe  that  should  there  be  a  loss  of  learning,  (1)  the  learning  begins 
anew,  arguably  at  the  same  rate;  and  (2)  it  continues  from  the  new  point  with  later  ships.  Both 
things  occurred  in  the  two  classes  we  analyzed.  Said  another  way,  loss  of  learning  acts  as  a 
reset  of  the  curve;  later  units  will  never  revert  to  the  old  curve ,  but  rather  proceed  from  the  reset 
unit  following  a  new  curve  with  the  old  slope.  This  last  belief  means  that  we  must  measure  loss 
of  learning  from  the  effective  unit  of  the  prior  ship,  moved  ahead  one  unit.  Thus,  we  must 
correct  out  all  other  effects  since  those  effects  (or  at  least  those  we  encountered)  are  either 
transient  or  fade  on  their  own  timescale.  Green  labor  appears  to  be  one  such  effect;  as  the 
labor  force  matures,  workers  approach  a  basic  efficiency. 

This  discussion  should  by  now  have  alerted  the  reader  as  to  how  complex  this  analysis 
is  and  how  dependent  it  is  on  each  step.  Get  one  step  wrong — we  know,  we  did  it  many  times 
as  we  groped  through  it — and  the  whole  thing  will  not  work.  One  can  “see”  the  effects;  to  prove 
them  analytically,  however,  requires  considerable  precision  and  care.  It  was  particularly 
confounding  that  for  many  of  the  ships  in  the  DDG  class,  the  Green  Labor  and  interval  effects 
tended  to  counteract  each  other.  This  was  because  when  time  between  ships  stretched  out — 
causing  loss  of  learning  due  to  interval  effect,  which  tended  to  drive  VL  MH  up — there  was  less 
work  in  the  yard,  so  manning  was  reduced  (largely  via  the  firing  of  Green  Labor),  resulting  in  a 
more  efficient  labor  force,  which  tended  to  drive  VL  MH  down.  At  other  times,  such  as  at  the 
onset  of  major  changes  in  a  class,  the  effects  can  reinforce  each  other  because  there  has  been 
a  gap,  but  work  then  suddenly  builds  up  in  the  yard.  It  was  not  until  both  effects  were  quantified 
that  this  complex  interaction  could  be  fully  understood.  This  interplay  has  doubtless  been  a 
major  complication  in  understanding  these  effects. 


35  Arena,  M.V.,  Blickstein,  I.,  Younossi,  O.,  Grammich.,  C.A.  (2006).  Why  has  the  cost  of  Navy  ships 
risen?  A  macroscopic  examination  of  the  trends  in  US  Naval  ship  costs  over  the  past  several  decades. 
RAND. 
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When  conducting  our  analysis,  we  first  determined  to  what  point  on  the  learning  curve 
each  ship  should  have  progressed  based  on  the  previous  ship.  This  was  conducted  iteratively 
from  the  second  ship  to  the  final  ship.  We  then  regressed  the  difference  between  the 
expectation  and  the  realization,  expressed  as  numbers  of  units  (or  fractions  thereof)  lost, 
against  days  between  ships.  The  regression  was  significant  at  the  95%  significance  level  (the 
“p-value”  was  0.0106).  The  coefficient  of  determination  (called  r2)  was  0.4324.  While  this  is  not 
an  overwhelming  r2,  it  is  a  respectable  one,  and  in  any  event,  the  determinant  of  validity  is 
significance,  not  r2. 

This  regression  shows  that  loss  of  learning  is  linear  with  days  between  ship  starts.  As 
ships  are  built  less  and  less  frequently,  learning  seems  to  “flatten.”  The  underlying  learning 
curve  slope,  however,  continues  to  remain  in  force.  The  ships  do  not  cease  to  learn,  they  lose 
learning.  This  is  an  important  distinction.  If  the  interval  shortens,  as  it  occasionally  does,  the 
MH  drop  faster,  reverting  to  the  original  learning  curve  slope  if  they  are  close  enough  together. 

There  is  much  to  discuss  about  this  finding,  but  first  we  should  observe  one  very 
important  fact.  The  underlying  learning  that  emerged  for  a  ship  class  of  25  ships  was  the  same 
learning  realized  by  the  first  nine  undisturbed  ships.  After  the  ninth  ship,  cost  began  to  rise,  so 
much  so  that  the  cost  of  the  thirteenth  ship  was  nearly  equal  to  the  cost  of  the  fourth  ship.  This 
may  not  be  surprising  after  all  our  descriptions  of  the  effects  that  change  costs.  The  point  is  that 
most  of  these  effects  do  not  happen  in  the  early  ships  because  they  tend  to  be  produced  at 
steady,  close,  intervals;  this  proved  true  for  both  DDGs  and  LHDs.  The  rise  that  began  in 
DDGs  after  nine  ships  (and  in  LHDs  after  four  ships,  though,  like  DDG,  about  five  years  into  the 
program)  was  caused  by  the  combined  effects  of  change  orders,  embedded  change  orders, 
labor  inefficiencies  caused  by  the  coincidence  of  a  new  class  of  large  ship  in  the  shipyard,  and 
loss  of  learning  due  to  the  delays  caused  by  the  incorporation  of  a  major  change  (DDG  51  Flight 
IIA).  We  are  confident  that  this  is  why  so  many  analysts  in  shipbuilding  cite  cessation  of 
learning;  but  we  wish  to  say  this  again  for  emphasis:  learning  continued  in  this  ship  class,  as  far 
as  we  had  data,  out  past  the  25th  ship.  The  reader  will  see  that  this  was  true  in  LHDs  as  well. 
Costs  rose  due  to  predictable  (or  at  least  expected)  effects,  and,  though  learning  was  lost  due  to 
slowdowns,  it  continued  on  from  each  point  “as  regular  as  sunrise.”  To  say  learning  stopped  is 
to  miss  the  point  and  to  forsake  the  opportunity  of  prediction. 

We  also  found  that  there  appeared  to  be  an  “ideal”  interval  between  ships  that  causes 
consistent  learning.  But  as  our  understanding  deepened,  we  began  to  appreciate  that  a  ship 
class  can  seem  to  take  any  learning  curve,  depending  on  the  predominant  interval  between 
ships.  We  will  also  show  the  days  between  ships  was  a  poor  way  to  display  the  interval,  and  we 
later  changed  to  percent  of  construction  duration;  this  change  promises  to  open  a  whole  new 
possibility.  We  will  we  discuss  both  things  further  below. 

The  Need  for  Confirmation:  The  LHD  Class 

The  DDG  model  has  been  laid  out.  In  many  important  analyses,  it  is  customary  to  do 
second  trials  (e.g.,  in  medical  studies).  The  second  test  guards  against  spurious  results.  We 
will  describe  the  meta-analysis  later.  In  this  case,  though,  a  confirming  analysis  was  needed 
more  than  in  usual  cost  analysis  cases  because  of  the  many  data  normalization  steps  that 
preceded  the  regression.  We  have  asserted  that  the  ultimate  regression  sustains  the  earlier 
steps,  but  we  cannot  escape  being  nervous  that  we  may  have  driven  towards  a  conclusion, 
inadvertently  forcing  the  conclusion.  This  is  natural  and  must  be  guarded  against.  Accordingly, 
we  immediately  set  about  repeating  the  analysis  for  a  second  ship  class. 
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We  did  exactly  the  same  analytical  steps  on  the  LHD  as  described  for  the  DDG  and 
arrived  at  exactly  the  same  conclusions;  we  found  the  LHD  class  had  done  just  what  the  DDGs 
did.  The  LHD  step  4  regression  for  the  Interval  Effect  was  significant  at  the  95%  significance 
level  (the  “p-value”  was  0.017).  The  coefficient  of  determination  (called  r2)  was  0.887.  This  r2 
was  higher,  due  simply  to  the  fact  that  the  LHDs  fell  into  two  groups;  3  units  lost  no  learning  and 
two  lost  all  learning.  Once  again,  the  determinant  of  validity  is  significance,  not  r2.  The  only 
difference  was  that  the  rate  at  which  the  LHDs  lost  learning  was  slower  (the  slope  of  the  line 
was  flatter);  and  the  “cross-over”  point,  the  interval  at  which  there  was  no  loss  of  learning  at  all, 
was  longer  in  days  (the  x-intercept  was  greater).  In  a  manner  of  speaking,  the  “time  constants” 
were  longer.  What  was  more  interesting,  however,  was  that  LHDs  had  on  two  occasions  lost 
learning  very  dramatically;  the  class  had  reverted  to  a  first  unit  (lost  all  learning)  twice,  at  LHD  5 
and  LHD  7.  The  underlying  learning  for  the  LHD  class  was  quite  similar  to  the  DDGs,  only  a  few 
percentage  points  flatter.  As  a  last  caution,  we  know  that  two  LHDs  lost  all  learning  and 
reverted  to  the  first  unit.  We,  thus,  cannot  know  whether  the  interval  effect  might  be  stronger 
than  we  have  evidence  for  because  these  two  ships  were  capped  at  the  first-unit  level. 
Accordingly,  we  know  that  the  LHD  interval  effect  is  at  least  as  strong  as  we  found,  and  quite 
likely  stronger. 

This  brings  us  to  an  important  point:  learning  due  to  interval,  according  to  our  model, 
occurs  in  units  with  time.  For  mature  classes  (say  after  nine  ships,  like  the  DDG),  the  loss  of  a 
few  units  is  not  a  large  number  of  MH,  because  the  curve  has  flattened  out  by  this  point  to  a  few 
percentage  points  per  unit.  For  less  mature  classes,  like  the  LHD,  however,  learning  is  still 
occurring  at  a  steep  rate,  and  the  loss  of  a  unit  may  be  more  on  the  order  of  10%  or  more.  In 
either  case,  if  the  Navy  or  the  Coast  Guard  have  negotiated  an  FPI  contract,  because  “follow 
ship  costs  should  be  well  known,”  and  there  is  a  delay  due  to  negotiations,  design  activity,  or 
lessened  SCN  budgets,  the  rise  can  erase  all  profit.  This  can  be  even  more  of  a  problem  if  the 
method  of  EAC  computation  does  not  catch  an  insidious  rise  in  preceding  ships,  perhaps  also 
due  to  factors  that  do  not  lie  at  the  feet  of  the  builder  and  which  will  cause  the  next  ship  to  rise 
for  additional  (additive)  reasons. 

Meta-analysis  and  Mutual  Confirmation 

To  summarize,  the  DDG  model  is,  as  far  as  statistics  can  take  us,  valid.  The  LHD  model 
is  also  valid,  and  we  can  use  either.  The  significance  of  the  entire  analysis  is  one  minus  the 
square  of  the  significance  of  each:  1  -  0.052  =  1  -0.0025  (significance  is  99.75%.)  This  well- 
known  technique  is  called  meta-analysis.  Most  importantly  though,  the  LHD  model  represents  a 
second  ship  class  and  was  undertaken  to  confirm  the  DDG  model.  Taken  together,  the  LHD 
and  DDG  models  are  much  stronger  than  either  is  alone  because  the  models  are  mutually 
confirming.  Taken  alone,  the  DDG  analysis  has  the  weaknesses  that  “first  models”  commonly 
have:  although  reasonable,  the  adjustments  were  taken  to  drive  towards  a  smooth  learning 
curve.  Statistically,  this  amounts  to  an  uncredited  loss  of  “degrees  of  freedom.”  Alternatively,  a 
hostile  view  can  arise  that  the  data  was  “cooked.”  The  LHD  model  alleviates  this  concern. 

Since  the  steps  taken  with  DDG  were  replicated  in  LHD  and  the  same  result  was  obtained,  our 
analysis  was  neither  dumb  luck  nor  manipulation.  Further,  the  LHD  class  had  only  7  ships 
completed,  so  taken  alone,  the  LHD  analysis  lacks  data  across  the  full  spectrum  of  interval 
length.  The  DDG  model  alleviates  this  concern  with  25  ships  and  a  wide  range  of  intervals. 
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ALM  2:  The  Enterprise  Model 

We  built  an  enterprise-level  model  that  instantiates  the  Advanced  Learning  Model  for  an 
entire  shipyard  complex,  namely  Northrop  Grumman  Ship  Systems.  We  call  this  a  forward- 
looking  model  because  it  projects  effects  given  the  results  of  the  prior  analysis,  which  we  call 
ALM  1 .  This  model,  called  ALM  2,  brought  about  three  significant  new  capabilities.  First,  the 
model  allows  for  the  quick  determination  of  the  impacts  of  the  effects  described  above  on  either 
a  future  ship  class  or  the  remainder  of  a  class.  It  also  allows  the  input  of  all  ships  in  the 
enterprise.  Second,  it  computes  the  labor  profile,  at  the  gross  level,  for  the  yard — a  capability 
which  formerly  took  weeks.  The  detailed  trade-by-trade  labor  profiles  are  still  done  by  hand, 
and  much  mischief  can  arise  at  the  trade  level  that  the  enterprise  model  is  blind  to;  but,  large 
what-ifs  can  nevertheless  be  done  quickly  as  a  first-order  approximation.  Third,  this  enterprise 
model  has  the  ability  to  model  cross-ship  effects  and,  since  it  is  in  closed  form,  feedback  effects 
on  labor. 

When  a  ship  cost  estimate  is  done,  the  apparent  labor  needs  are  determined  for  a 
notional  labor  profile — the  profile  that  was  in  effect  at  the  time  of  the  historical  data.  The  model 
first  needs  to  compute  the  efficiency  of  the  labor  force  in  the  yard  where  it  is  being  built  based 
on  the  mix  of  Green  Labor.  The  model  then  will  compute  interval  effects  based  on  the 
preceding  ship  and  the  interval  before  the  ship  in  question.  At  this  point,  the  model  will  adjust 
the  need  for  labor.  This  adjustment  changes  the  labor  mix  in  the  yard  because  new  labor  must 
be  hired  above  (or  below)  what  was  expected,  due  to  the  effects  of  these  two  variables.  This 
produces  a  feedback  loop  which  happily  converges  fairly  quickly.  Likewise,  the  model,  having 
all  ships  in  the  yard  loaded  in,  quickly  computes  the  effects  of  the  new  labor  on  all  the  other 
ships  in  the  yard  as  the  labor  force,  and  the  associated  feedback  loops,  swells  or  shrinks  and 
does  so  quickly. 

The  enterprise  model  brought  out  an  understanding  that  is  worthy  of  some  discussion. 
As  we  began  to  use  the  model  to  predict  ships,  we  needed  to  develop  a  good  prediction  for 
change  orders.  When  we  did  the  analysis  using  the  carefully  deconstructed  change  order  data 
for  these  two  ship  classes,  we  were  taken  aback  by  the  size  and  the  implications  of  what  we 
found.  We  found  that  change  orders  were  large  on  the  first  ship,  which  we  had  known,  and 
averaged  12.30%  of  base  costs  (with  a  wide  variability).  We  knew  that  by  definition  change 
orders  would  be  large  at  Flight  change  or  at  major  modifications,  so  when  we  found  that 
changes  were  8.62%  of  cost  it  was  sobering  but  not  surprising.  What  surprised  us  was  that 
change  orders  on  the  rest  of  the  ships,  ships  that  were  neither  first  ships  nor  major 
modifications,  was  2.97%,  with  a  fairly  clear  pattern  of  rising  over  the  class  but  virtually  always 
exceeding  1 .2%.  This  change-order  pattern  meant  that  by  the  25th  ship,  with  two  major 
modifications,  the  labor  cost  of  the  DDG  change  orders  had  have  become  equal  to  about  half 
that  of  the  base  ship.  On  the  LHD,  change  orders  have  risen  to  be  about  one-quarter  of  the 
cost  of  the  base  work  by  the  7th  ship.  This  finding  showed  us  that  change  orders  are  a  much 
larger  part  of  the  cost  of  follow  ships  than  we  had  realized,  even  though  we  knew  the 
conventional  wisdom  that  changes  are  a  big  factor  in  ship  costs. 

It  should  not  be  concluded  that  change  orders  are  inherently  bad.  Change  orders  are  at 
least  partly  a  response  to  lessons  learned,  changes  in  technology  and  in  the  threat.  That  said, 
they  bear  careful  monitoring  due  to  their  size,  which  begins  to  rival  initial  cost  in  their  claim  on 
scarce  SON.  Change  orders  also  have  a  second,  hitherto-less-well-understood  effect  on  ship 
costs.  It  is  quite  likely  that  change  orders  act  both  as  chicken  and  egg  in  the  ship-cost  scenario. 
As  budgets  drop,  interval  increases,  allowing  an  ever-increasing  pressure  for  change — be  it  due 
to  an  industrious  engineering  community,  technology  changes,  or  requirements  changes. 
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Conversely,  changes  cause  design  delay,  increasing  intervals,  sometimes  considerably,  as  they 
do  in  major  modifications.  Thus,  changes  are  already  a  considerable  factor  and  have  the  ability 
to  become  more  of  a  factor  in  tight  budget  years,  as  well  as  to  drive  the  other  effects  to  become 
larger  as  they  delay  ships  and  contribute  to  cost  rises — which  destabilize  labor  costs. 

Implications 

At  the  outset  of  this  paper,  we  described  how  ship  costs  have  suddenly  seemed  to  go 
out  of  control  just  as  budgets  have  dropped,  exacerbating  the  problem.  There  are  no  doubt 
many  reasons  for  such  a  problem,  some  of  which  are  outlined  in  the  RAND  study  already 
referenced.  Many  of  the  reasons  cited  in  the  RAND  paper,  such  as  inflation  and  commodity 
volatility,  have  conventional  means  of  prediction,  however  flawed  the  results  may  be.  Our 
paper,  on  the  other  hand,  puts  forward  three  new  effects  that  were  hitherto  generally  ignored 
analytically  or  discounted  by  decision-makers.  As  we  have  already  said,  change-order 
estimates  in  general  turned  out  to  be  quite  accurate;  without  this  accuracy,  the  analysis  would 
have  been  impossible.  The  “lost”  change  orders  are  unarguable  in  principle,  and  although  the 
exact  values  estimated  could  be  quibbled  with,  they  are  certainly  “generally  right.”  The 
illumination  we  have  brought  to  this  issue  will  help  explain  past  cost  increases.  And  closer 
attention  to  tracking  change  orders  going  forward  should  be  a  big  help  to  future  cost  estimation. 
The  Green  Labor  model  we  described  was  in  use  at  one  Northrop  Grumman  shipyard  and  is 
now  in  use  at  them  all,  although  its  impact  is  debated  and  occasionally  somewhat  discounted. 
We  believe  that  Green  Labor’s  part  in  this  analysis  will  go  far  to  bolster  the  basis  for  the 
computation  of  this  effect.  The  interval  effect  has  always  been  part  of  shipyard  lore,  but  has 
never  been  quantified.  Even  now,  shipyards  are  struggling  to  understand  the  model  and  to 
overcome  past  “explanations”  for  rising  costs,  such  as:  labor  inefficiencies  (true,  but  overstated 
for  the  lack  of  an  agreed-upon  model),  one-time  events  (again,  true,  but  arguably  overstated  for 
lack  of  sufficiently  reliable  discrete  cost  tracking),  and  disruption  effects  such  as  were  alleged  to 
have  happened  twice  on  DDG  and  once  on  LHD  (we  found  no  significant  differences  that 
needed  explanation,  so  the  supposed  disruption,  although  appealing  intellectually,  was  absent  if 
you  believe  our  analysis). 

The  net  of  the  effects  in  this  paper  is  extremely  large,  and  almost  all  was  cost  growth. 
The  newly  explained  cost  growth  in  labor,  resulting  from  embedded  change  orders.  Green  Labor 

and  loss  of  learning  due  to  interval  for  all  ships  from  12  through  25  of  the  DDG  is  a  total  of  21%, 

and  for  all  ships  from  3  through  7  of  the  LHD  is  a  total  of  19%.  This  is  a  staggering  amount  of 
cost  to  have  been  hitherto  unexplained.  Of  the  newly  explained  cost  growth  in  labor,  8.8% 
(almost  half)  of  the  growth  in  DDGs  was  from  loss  of  learning  due  to  the  interval  effect;  for  the 
LHD  class,  14%  (almost  3A)  was  from  loss  of  learning  due  to  the  interval  effect.  Most  of  the  rest 
of  the  unexplained  cost  growth  was  due  to  “lost”  change  orders.  The  DDG  interval  effect  was 
less  of  the  total  partly  because  the  class  was  more  mature  and  further  down  the  learning  curve; 
thus,  units  lost  were  smaller  in  MH  than  for  LHDs.  Conversely,  for  LHDs,  change  orders  were 
less  of  the  total  because  there  had  been  fewer  units  for  them  to  have  accumulated.  Green  labor 
was  the  least  of  the  effects,  and  tended  to  be  pluses  or  minuses  as  labor  waxed  and  waned. 

We  thus  believe  that  a  large  mystery  has  been  solved,  and  that  we  have  provided  the  ability  to 
avoid  similar  large  errors  in  the  future.  As  with  any  cost-estimating  technique,  of  course,  the 
ability  to  project  will  depend  upon  how  well  we  can  determine  the  inputs. 

This  model  has  the  capability  to  account  for  many  large  and  unpleasant  cost-growth 
effects,  especially  for  modified  ships  that  are  characterized  by  long  intervals,  large  change-order 
packages,  and  large  swings  in  labor  as  the  class  goes  back  into  production.  Hence,  we  believe 
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that  the  model  will  be  invaluable  in  better  predicting  costs.  We  hope  it  will  be  helpful  in 
reducing  mistrust  and  tension  between  the  Navy  and  the  Coast  Guard  on  the  one  hand,  and  the 
shipyards  on  the  other — as  well  as  between  the  Congress  and  the  services.  We  believe  the 
model  goes  a  long  way  towards  explaining  cost  rises  in  terms  the  various  parties  can  better 
understand  as  unavoidable  in  the  past  and  more  predictable  in  the  future. 

There  are  policy  implications  to  these  findings  as  well.  We  believe  costs  can  be  better 
estimated  now;  but  beyond  that,  they  can  be  better  controlled  as  well.  One  simple  exercise 
involved  changing  the  acquisition  profile  for  Large  Deck  Amphibious  ships.  By  optimizing  the 
timing  of  ship  acquisitions,  we  found  that  labor  costs  could  be  reduced  by  almost  6%  in  constant 
dollars — and  this  was  with  only  a  single  ship  class  to  work  with.  In  addition,  in  order  to  be 
successful,  the  Navy’s  current  cost  reduction  strategy  of  re-using  existing  hull  forms  for  new 
classes  of  ships  must  obey  the  constraints  illuminated  here.  Change  orders  must  be 
suppressed  to  the  degree  that  they  can,  intervals  between  ships  must  be  thoughtfully  timed,  and 
demand  variance  that  results  in  workforce  instability  must  be  avoided.  We  note  that  this  is 
consistent  with  an  enterprise  view  of  shipbuilding:  the  Navy  making  acquisition  decisions  that 
support  its  private-industry  partners  in  their  efforts  to  contain  costs.  On  the  other  hand,  for  the 
Navy  to  pursue  acquisition  strategies  that  ignore  the  demonstrated  effects  of  change  orders, 
loss  of  learning  due  to  interval,  and  Green  Labor — and  at  the  same  time  to  press  for  cost 
reduction — is  inherently  problematic.  It  is  further  setting  the  enterprise  up  for  failure. 


Next  Steps 

We  hope  to  take  this  analysis  to  other  ship  classes.  We  further  hope  to  take  it  beyond 
ships  to  other  commodities.  As  strange  as  it  may  sound,  application  to  satellites  is  quite 
promising,  as  they  are,  like  ships,  complex  objects  with  long  development  spans  involving 
integration  of  many  components,  operated  in  demanding  environments,  and  procured  in  small 
“batches”  (usually  one  or  two).  There  is  no  clear  reason  why  this  methodology  should  not  work 
for  other  sorts  of  production.  Indeed,  the  problem  may  be  considerably  easier  since  labor  force 
changes  should  be  less  of  an  effect  in  items  of  shorter  duration  and  higher  production  rate. 
Change  orders  may  also  be  easier,  because  data  will  not  be  so  old  in  calendar  terms  and  since 
the  shorter  times  lend  themselves  to  fewer  changes. 

We  would  like  to  point  out  that  so-called  “rate  effects”  in  learning  have  been  elusive.  We 
know  of  no  demonstration  of  a  statistically  significant  rate  effect.  We  suggest  that  perhaps  the 
“rate  effect,”  which  is  commonly  introduced  as  a  second  term  in  the  learning  equation,  may 
simply  be  an  incorrect  portrayal  of  the  interval  effect  we  have  demonstrated.  Rate  effect  uses 
the  number  of  units  to  be  produced  in  a  given  year  and  lowers  costs  as  the  quantity  rises  and 
raises  costs  as  it  drops.  This  acts  to  reduce  cost  when  units  are  built  closer  together  (more 
production  units  per  unit  time),  thus  mirroring  the  sense  of  the  interval  effect  we  found  but  using 
different  mathematical  expressions  to  model  it.  Statistics  have  the  unfortunate  limitation  that  a 
hypothesis  must  be  formulated  for  the  test.  If  the  hypothesis  is  structured  in  an  incorrect  form, 
the  model  is  unlikely  to  be  significant;  thus,  the  hypothesis  that  the  model  is  invalid  cannot  be 
rejected.  Many  good  ideas  founder  on  this  shoal,  and  this  failure  of  an  “almost  right  model”  or  a 
model  that  acts  in  a  sensible  way,  but  is  formulated  a  bit  incorrectly,  frustrates  the  analyst  and 
leads  to  unreasonable  distrust  of  statistics.  We  look  forward  to  investigating  the  rate  effect 
relative  to  the  interval  effect  and  believe  that  the  interval  effect  will  replace  it  in  practice. 

As  we  have  alluded  to,  we  have  already  begun  a  re-formulation  of  the  model.  We  have 
changed  the  expression  of  the  model  for  loss  of  learning  from  days  to  percent  of  construction 
duration.  This  is  merely  a  change  in  scale,  and  the  results  do  not  change  for  either  model. 
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What  does  change,  however,  is  that  when  the  model  is  expressed  in  the  new  way,  we  discover 
that  the  LHD  and  DDG  graphs  of  loss  of  learning  overlay  each  other.  In  the  “days  form,”  we  had 
no  way  to  move  to  another  class,  because  we  had  no  way  to  interpolate  save  linearly,  and  we 
were  wary  of  this  assumption.  Worse  yet,  when  applied  to  a  much  smaller  ship,  like  a  frigate,  or 
a  much  larger  ship,  like  an  aircraft  carrier,  the  extrapolation  became  irrational.  This  amounted 
to  reductio  ad  absurdum  of  linear  extrapolation  and  so  cast  doubt  upon  interpolation.  When 
expressed  as  percent  of  interval,  however,  the  result,  although  not  yet  tested,  is  appealing  and 
intuitive.  This  version  of  the  model  is  called  ALM  3. 

We  are  not  shipbuilders,  and  further,  we  are  always  reluctant  to  speculate  beyond  what 
we  can  prove.  Nonetheless,  one  of  the  reasons  we  are  inclined  to  believe  this  “percent  form”  is 
that  the  nature  of  work  on  a  large  ship  changes  less  quickly  during  production  than  for  small 
ships.  For  example,  shell  plating  and  other  large-object  welding  lasts  a  similar  percentage  of 
construction  time  on  large  ships  as  it  does  for  smaller  ships;  hence,  we  speculate  that  the 
opportunity  to  “learn”  lasts  longer,  and  time  to  lose  it  takes  longer  in  days  but  not  in  percent. 

This  change  in  scale  has  taken  us  to  another  point.  We  are  in  the  midst  of  trying  to 
determine  whether  the  percent  model  will  lead  us  to  a  long-sought  capability:  the  capability  to 
predict  the  underlying  learning  curve  for  any  ship  knowing  only  the  expected  construction 
duration  and  the  planned  schedule  for  the  class.  We  anticipate  having  a  preliminary  version  of 
this  in  time  for  the  symposium,  or  to  have  learned  that  this  is  un-doable.  If  we  can  achieve  this, 
we  feel  the  results  will  be  every  bit  as  important  as  the  loss-of-learning  model.  It  would  also 
increase  the  importance  of  both  accurate  schedule  estimating  and  stable  program  schedules,  as 
these  are  key  inputs  to  the  learning-curve  determination,  which  in  turn  is  the  single  biggest 
driver  of  labor  cost.  It  is  premature  to  make  a  claim  of  broader  application,  especially  since  the 
ship  portion  is  as  yet  undone,  but  we  cannot  stop  our  minds  from  racing  ahead.  We  hold  out 
hope  for  this  model  being  useful  in  other  commodities.  We  have,  on  an  informal  basis,  noted 
that  the  interval  for  Global  Hawk  is  similar  to  that  for  DDGs  in  percent,  and  that  the  learning 
curve  is  similar.  We  raise  this  point  in  order  to  excite  the  reader’s  interest  and  to  indicate  that 
the  possibility  of  further  application  is  real. 

Conclusions 

This  paper  is  short,  and  so  our  conclusions  should  be  short  as  well.  We  have  shown 
that  a  careful  treatment  of  change  orders,  labor  force  mix,  and  time  between  ship  starts  has 
enabled  us  to  demonstrate — we  believe  for  the  first  time — a  closed-form  and  statistically 
rigorous  treatment  of  loss  of  learning.  The  analysis  has  validated  an  old  labor  mix  model  and 
shown  it  to  have  considerably  credibility,  and  has  unified  change  orders,  labor  profiles,  and  loss 
of  learning  in  a  way  that  substantially  changes  ship-cost  estimation.  We  have  already  created 
an  enterprise  model  and  have  hopes  of  being  able  to  extend  the  model  to  other  commodities. 
We  will,  perhaps,  even  be  able  to  predict  learning  curves  for  ship  classes — a  capability  that 
does  not  yet  exist. 
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Panel  18  -  Modeling  &  Simulation  in  Support  of  Acquisition 


Thursday, 
May  17,  2007 


1:45  p.m.-  Chair: 
3:15  p.m. 


Daniel  A.  Nussbaum,  Professor,  Naval  Postgraduate  School 

Papers: 

Simulation  Based  Acquisition  Revisited 

Michael  F.  McGrath,  Deputy  Assistant  Secretary  of  the  Navy  for  Research, 
Development,  Test  and  Evaluation 

An  Innovative  Approach  for  Training  Acquisitions — Part  II 

Fred  Hartman,  Deputy  Director,  Readiness  and  Training,  and  Director  of 
the  Joint  Assessment  and  Enabling  Capability,  Office  of  the  Under 
Secretary  of  Defense  for  Personnel  and  Readiness 

An  Open  Strategy  for  the  Acquisition  of  Models  and  Simulations 

Rudy  Darken,  Naval  Postgraduate  School 

Application  of  Systmes  Engineering  Principles  in  Curricular  Design 

David  H.  Olweii,  Jarema  M.  Didoszak,  Jean  M.  Catalano,  Naval 
Postgraduate  School,  and  LCDR  Joseph  Cohn,  USN,  Medical  Service 
Corps 


Chair:  Daniel  A.  Nussbaum,  PhD  is  a  Professor  at  the  Naval  Postgraduate  School,  in  the  Operations 
Research  department,  in  Monterey,  California.  His  expertise  is  in  cost/benefit  analyses,  life  cycle  cost 
estimating  and  modeling,  budget  preparation  and  justification,  performance  measurement  and  earned 
value  management  (EVM),  activity  based  costing  (ABC)  and  Total  Cost  of  Ownership  (TCO)  analyses. 

From  December  1999  through  June  2004  he  was  a  Principal  with  Booz  Allen  Hamilton,  providing 
estimating  and  analysis  services  to  senior  levels  of  the  US  Federal  government.  He  has  been  the  chief 
advisor  to  the  Secretary  of  Navy  on  all  aspects  of  cost  estimating  and  analysis  throughout  the  Navy,  and 
has  held  other  management  and  analysis  positions  with  the  US  Army  and  Navy,  in  this  country  and  in 
Europe. 

In  a  prior  life,  he  was  a  tenured  university  faculty  member. 

Dr.  Nussbaum  has  a  BA,  in  Mathematics  and  Economics  from  Columbia  University,  and  a  Ph.D.,  in 
Mathematics  from  Michigan  State  University.  He  has  held  post  doctoral  positions  in  Econometrics  and 
Operations  Research,  and  in  National  Security  Studies  at  Washington  State  University  and  Harvard 
University. 

He  is  active  in  professional  societies,  currently  serving  as  the  President  of  the  Society  of  Cost  Estimating 
and  Analysis.  He  has  previously  been  the  VP  of  the  Washington  chapter  of  INFORMS,  and  he  has 
served  on  the  Board  of  the  Military  Operations  Research  Society. 
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He  publishes  and  speaks  regularly  before  professional  audiences. 


Finally,  he  is  married,  has  two  children  and  four  grandchildren.  He  is  a  lap  swimmer  and  a  dedicated  herb 
and  vegetable  gardener. 

Daniel  A.  Nussbaum,  Professor 

Graduate  School  of  Operational  and  Information  Sciences 
Monterey,  CA  93943 
Phone:  831-656-2387 
Email:  dnussbaum@nps.edu 
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Simulation  Based  Acquisition  Revisited 


Presenter:  Michael  F.  McGrath,  Deputy  Assistant  Secretary  of  the  Navy  for  Research,  Development, 
Test  and  Evaluation.  Dr.  Michael  McGrath  is  the  Deputy  Assistant  Secretary  of  the  Navy  for  Research, 
Development,  Test  and  Evaluation.  His  role  is  to  develop  programs  to  bridge  the  gap  in  transitioning  from 
Science  and  Technology  to  Acquisition,  and  to  improve  processes  for  integrating  T&E  across  acquisition 
programs.  Prior  to  his  appointment  to  this  position  in  2003,  Dr.  McGrath  spent  five  years  as  Vice 
President  for  Government  Business  at  the  Sarnoff  Corporation,  a  leading  R&D  company  with  both 
commercial  and  government  clients.  Dr.  McGrath  has  28  years  of  prior  government  experience,  including 
logistics  at  NAVAIR  in  the  1970s,  acquisition  in  OSD  in  the  1980s,  and  technology  development  at 
DARPA  in  the  1990s.  Dr.  McGrath  holds  a  BS  in  Space  Science  and  Applied  Physics,  a  MS  in  Aerospace 
Engineering,  and  a  doctorate  in  Operations  Research  from  George  Washington  University. 
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An  Innovative  Approach  for  Training  Acquisitions — Part  II 


Presenter:  Fred  Hartman,  Deputy  Director,  Readiness  and  Training,  and  Director  of  the  Joint 
Assessment  and  Enabling  Capability,  Office  of  the  Under  Secretary  of  Defense  for  Personnel  and 
Readiness.  Hartman  is  a  distinguished  member  of  the  DoD  M&S  Community,  and  has  served  for  many 
years  in  various  government  and  industry  positions.  He  is  currently  the  Deputy  Director  of  Readiness  and 
T raining  Policy  and  Programs  (RTPP)  for  the  Office  of  the  Under  Secretary  of  Defense,  (Personnel  and 
Readiness)  and  the  Director  of  Training  Transformation  Joint  Assessment  and  Enabling  Capability 
(JAEC).  Mr.  Hartman  also  serves  as  a  tri-chair  and  Training  Community  member  for  the  DoD  M&S 
Steering  Committee. 

Mr.  Hartman  was  commissioned  in  Field  Artillery  on  graduation  from  the  United  States  Military  Academy 
and  later  earned  a  Master  of  Science  degree  in  Operations  Analysis  from  the  Naval  Postgraduate  School. 
In  addition  to  FA  and  Aviation  assignments,  he  served  as:  military  operations  analyst,  procurement 
programs  analyst,  executive  assistant  and  analyst  for  the  Deputy  Under  Secretary  of  the  Army  for 
Operations  Research. 

After  his  Army  career,  Mr.  Hartman  was  an  Executive  Vice  President  for  CACI.  He  also  served  as  Chief 
Operating  Officer,  co-founder  and  board  member  for  Applied  Solutions  International,  Inc.  In  1995,  Mr. 
Hartman  joined  the  Institute  for  Defense  Analyses  as  a  modeling  and  simulation  consultant  to  the  Deputy 
Under  Secretary  of  Defense  for  Readiness.  In  2004,  he  assumed  duties  as  Director  of  the  Joint 
Assessment  and  Enabling  Capability  and,  in  2005,  took  on  the  additional  responsibilities  as  Deputy 
Director  of  RTPP.  Mr.  Hartman  has  served  as  a  member  of  the  Army  Science  Board,  led  a  study  panel  for 
the  National  Academy  of  Sciences  Board  on  Army  Science  and  Technology  and  is  a  past  president  and 
fellow  of  the  Military  Operations  Research  Society. 


Abstract 

The  Training  Capabilities  Analysis  of  Alternatives  (TC  AoA)  published  in  2004  examined 
a  host  of  issues  related  to  training  transformation  and  the  future  of  training  to  meet  joint 
operational  needs.  During  the  conduct  of  the  TC  AoA,  the  industry  partners  participating  in  a 
business  game  indicated  the  DoD  should  improve  their  approach  for  buying  training  simulations 
and  tools  as  well  training  support  services.  The  AoA  recommended  a  limited  acquisition 
prototype  (known  as  Alternative  #4)  to  explore  the  many  issues  for  procurement  of  training, 
including  privatization  of  both  training-support  and  training-tool  development  efforts  with 
competitive  market  forces.  The  RAND  Corporation  conducted  a  follow-on  research  effort  in 
2005  to  produce  and  implement  an  evaluation  plan  for  the  prototype  of  this  alternative.  The 
current  prototype  is  being  executed  this  year  to  include  the  Defense  and  Inter-agency 
Coordination  for  Homeland  Defense  training  issues. 

This  presentation  should  be  of  interest  to  those  participating  in,  or  charged  with  carrying 
out,  innovative  approaches  for  Defense  Acquisitions,  including  Modeling  and  Simulation.  After  a 
short  update  on  the  training  community  progress  on  the  Alternative  #4  business  process,  the 
common  and  cross-cutting  aspects  of  this  prototype  from  training  to  other  functional  DoD 
applications  will  be  discussed. 
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An  Open  Strategy  for  the  Acquisition  of  Models  and  Simulations 


Presenter:  Rudolph  Darken,  Director,  the  MOVES  Institute,  Naval  Postgraduate  School.  Darken  is  the 
Director  of  the  Defense  Modeling  and  Simulation  Institute  (MOVES)  and  Professor  of  Computer  Science 
at  the  Naval  Postgraduate  School  in  Monterey,  California.  He  is  also  the  Associate  Director  for  Research 
for  the  Center  for  Homeland  Defense  and  Security  and  serves  on  the  Academic  Committees  for  Modeling 
and  Simulation  (MOVES)  as  well  as  Human  Systems  Integration  (HSI).  His  personal  research  has  been 
primarily  focused  on  human  factors  and  training  using  virtual  environments  and  computer  gaming  media 
with  emphasis  on  navigation  and  wayfinding  in  large-scale  virtual  worlds.  He  is  a  Senior  Editor  of 
PRESENCE  Journal,  the  MIT  Press  journal  of  teleoperators  and  virtual  environments.  He  received  his  BS 
in  Computer  Science  Engineering  from  the  University  of  Illinois  at  Chicago  in  1 990  and  his  MS  and  DSc 
degrees  in  Computer  Science  from  The  George  Washington  University  in  1993  and  1995,  respectively. 

Rudolph  P.  Darken 
Director,  The  MOVES  Institute 
Naval  Postgraduate  School 
darken@nps.edu 


Abstract 

It  remains  unknown  exactly  how  much  the  Department  of  Defense  spends  on  Modeling 
and  Simulation  annually.  Estimates  range  from  $9B  to  $15B,  but  precise  numbers  are 
unavailable  because  it’s  hard  to  say  where  M&S  starts  and  ends  in  a  training,  analysis,  or 
acquisition  program.  But  if  these  estimates  are  anywhere  near  accurate,  a  discussion  of  how  we 
acquire  M&S  capabilities  and  how  we  can  get  more  done  with  less  is  warranted.  The  DoD  has 
recently  approved  the  equal  consideration  of  open  source  for  supplying  software  for  DoD 
functions.  M&S,  being  largely  a  software-based  technology,  should  fall  under  this  policy. 
However,  M&S  remains  an  almost  exclusively  proprietary  domain  for  software  development — 
even  though  the  commonalities  between  M&S  products  are  vast.  The  simple  fact  is  that  the 
current  model  for  the  acquisition  of  M&S  products  requires  that  the  government  pay  for  the 
same  capabilities  over  and  over  because  reusability  is  not  considered,  or  at  least  not  considered 
to  be  possible  or  practical.  This  is  a  myth.  There  are  good  examples  in  which  this  is  changing — 
for  example,  reusable  terrain  databases  in  aviation  simulation.  The  saving  here  are  enormous. 
But  more  M&S  products  are  bought  and  paid  for  as  complete  products,  where  the  supplier  is 
virtually  guaranteed  downstream  revenue  due  to  “vendor  lock-in”  common  in  software  products. 
There  is  a  way  out.  An  open  source  and  standards  framework  to  simulation  that  modularizes 
M&S  software  so  that  the  government  buys  only  what  it  needs  is  possible.  In  fact,  we  have  an 
excellent  example  of  such  a  framework  from  enterprise  computing  called  the  Service  Oriented 
Architecture  (SOA).  This  presentation  will  introduce  concepts  of  open  source  and  standards  for 
the  Defense  M&S  community  and  will  show  how  a  SOA  approach  (but  not  a  literal  SOA 
implementation)  is  possible  and  can  result  in  extremely  large  efficiency  gains  due  to  reusable 
software,  content,  and  source  code. 
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Application  of  Systems  Engineering  Principles  in  Curricular  Design 


Presenter:  David  H.  Olwell  is  the  Chairman  of  the  Department  of  Systems  Engineering  at  the  Naval 
Postgraduate  School.  Dr.  Olwell  previously  taught  at  the  United  States  Military  Academy  in  the 
Department  of  Mathematical  Sciences  and  in  the  Operations  Research  Department  at  the  Naval 
Postgraduate  School.  His  specific  research  interests  focus  on  the  analysis  of  systems,  with  emphasis 
on  reliability,  quality,  and  warranty  issues. 

Author:  Jarema  M.  Didoszak  is  a  Research  Assistant  Professor  in  the  Mechanical  and  Astronautical 
Engineering  Department  at  the  Naval  Postgraduate  School,  Monterey,  California.  Mr.  Didoszak  is  also  an 
engineering  duty  officer  in  the  Navy  Reserve.  Over  the  last  four  years,  he  has  been  conducting  NAVSEA- 
funded  modeling  and  simulation  research  in  support  of  the  DDG  81,  LPD  17,  DDG  1000  and  LCS  ship 
shock  trial  programs.  He  is  currently  pursuing  a  PhD  in  Mechanical  Engineering. 

Author:  Jean  M.  Catalano  is  a  Research  Assistant  in  the  Systems  Engineering  Department  at  the  Naval 
Postgraduate  School,  Monterey,  California.  After  serving  on  active  duty  in  the  US  Navy,  she  supported 
the  NAVSEA  Warfare  Systems  Engineering  Directorate  (NAVSEA06)  before  coming  to  Naval 
Postgraduate  School,  where  she  is  currently  pursuing  a  PhD  in  Software  Engineering. 

Authors:  LCDR  Joseph  Cohn,  USN,  Medical  Service  Corps,  is  an  Aerospace  Experimental  Psychologist 
in  the  US  Navy’s  Medical  Service  Corps.  He  has  served  as  the  Lead  Scientist  on  a  range  of  training 
programs,  overseeing  the  design,  development,  validation  and  transition  of  numerous  Virtual  Environment 
training  systems  across  the  DoN.  He  is  currently  serving  as  the  Lead  Scientist  as  part  of  the  Chief  of 
Naval  Personnel’s  Strategic  Concepts  group,  focusing  on  defining  future  challenges  and  solutions  to 
Navy  Manpower,  Personnel,  Training  and  Education. 


Abstract 

The  Defense  Modeling  and  Simulation  Coordination  Office  presented  the  Naval 
Postgraduate  School  with  an  enormous  challenge  in  2006:  design  and  deliver  an  educational 
program  by  2008,  for  20,000  or  more  acquisition  professionals,  focusing  on  the  effective  use  of 
modeling  and  simulation  in  acquisition.  The  acquisition  workforce  is  central  to  force 
transformation,  and  education  is  the  key  to  transforming  that  workforce.  This  paper  describes 
the  processes,  lessons  learned  to  date,  and  assessment  plan  for  this  project. 

We  applied  a  systems-engineering  approach  to  the  problem  of  curricular  design.  The 
resulting  solution  consists  of  four  spirals.  The  first  spiral  focused  on  defining  the  problem.  We 
developed  our  analysis  based  on  factors  such  as  the  market  segmentation  of  the  acquisition 
workforce,  the  current  resources  available,  the  state  of  the  modeling  and  simulation  body  of 
knowledge,  the  desired  educational  outcomes  for  each  market  segment,  and  the  gaps  that 
existed  between  those  outcomes  and  the  existing  resources.  At  each  step  in  the  process,  we 
involved  key  stakeholders  from  the  acquisition,  test  and  evaluation  and  training  communities. 
We  describe  the  results  of  this  process. 

In  the  second  spiral,  our  goal  is  to  construct  a  learning  architecture  to  cover  the  gaps 
identified  in  the  first  spiral.  We  describe  the  course  content,  scope,  and  delivery  methods  that 
are  proposed  based  on  those  needs  from  the  first  spiral. 

The  results  of  the  first  spiral,  initiation  of  spiral  two  and  subsequent  lessons  learned,  will 
be  the  focus  of  our  discussion  herein.  We  will  also  briefly  summarize  the  third  and  fourth 
spirals,  which  involve  course  design  and  testing  in  spiral  three  and  delivery  and  assessment  of 
the  curriculum  for  spiral  four. 

Keywords:  Modeling  &  Simulation,  Education,  Acquisition,  Systems  Engineering. 
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Panel  19  -  Acquisition  of  Services 


Thursday, 
May  17,  2007 


1:45  p.m.-  chair: 
5:00  p.m. 


Stuart  A.  Hazlett,  Deputy  Director  for  Strategic  Sourcing,  Defense 
Procurement  and  Acquisition  Policy, 


Papers: 


Enterprise  Challenges  Facing  the  Strategic  Sourcing  of  Services 

Stuart  Hazlett,  Deputy  Director,  Strategic  Sourcing,  Defense  Procurement 
and  Acquisition  Policy 

The  Commander  of  Naval  Installations  and  the  Acquisition  of  Services 

Thomas  Trump,  Assistant  Deputy  Commander  for  Contracting 
Management,  Naval  Supply  Systems  Command 

Performing  Quality  Assurance  and  Effectively  Administering  the  Strategic 
Sourcing  of  Services 

COL  Anthony  Incorvati,  Chief,  BTA  Division  and  Strategic  Sourcing, 
ITEC4,  Army  Contracting  Agency 

Essential  Elements  to  Successful  Execution  of  Strategically  Sourced 
Services 

Randall  McFadden,  Director,  Acquisition  Management  and  Integration 
Center 

Delivering  to  the  Warfighter 

Dale  Huegen,  Deputy  Chief,  Command  Acquisition,  USTRANSCOM 

Managing  the  Service  Supply  Chain  in  DoD:  Implications  for  the  Program 
Management  Infrastructure 

Rene  Rendon  and  Uday  Apte,  Naval  Postgraduate  School 


Chair:  Stuart  A.  Hazlett,  Deputy  Director  for  Strategic  Sourcing,  Defense  Procurement  and  Acquisition 
Policy,  serves  as  an  advisor  to  the  Director,  Defense  Procurement  and  Acquisition  Policy  (DPAP)  and  his 
staff  on  matters  relating  to  the  Acquisition  and  Sourcing  Processes  and  initiatives  across  the  Department 
of  Defense.  Mr.  Hazlett  provides  advice  and  counsel  regarding  the  formulation  and  development  of  policy 
on  various  strategic  sourcing  programs  associated  with  transformation,  and  leads  a  focused  team  that 
oversees  critical  Acquisition,  Technology  &  Logistics  (AT&L)  transformation  initiatives  that  accelerate 
delivery  of  effective  and  efficient  business  mission  functions. 

Before  taking  his  current  position,  Mr.  Hazlett  was  the  Chief  of  Procurement  Transformation,  Office  of  the 
Deputy  Assistant  Secretary  for  Contracting,  Assistant  Secretary  of  the  Air  Force  for  Acquisition, 
Headquarters  U.S.  Air  Force,  where  he  was  charged  with  improving  agile  sourcing  in  the  Air  Force.  His 
office  maintained  a  contracting  strategic  management  framework  to  identify,  develop,  and  field  strategic 
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initiatives  to  support  a  contracting  enterprise  responsible  for  executing  $61 B  annually.  Mr.  Hazlett  entered 
federal  service  in  1984  as  a  Pacer  Produce  assigned  to  the  San  Antonio  Air  Logistics  Center  at  Kelly  Air 
Force  Base,  Texas,  and  has  held  contracting  offer  warrants  in  the  areas  of  research  and  development, 
systems,  and  sustainment  acquisitions  in  support  of  numerous  major  weapons  systems.  During  his 
career,  he  has  served  in  direct  support  of  congressionally  mandated  panels  dealing  with  acquisition 
issues  facing  the  federal  government. 

Mr.  Hazlett  holds  a  Bachelor  of  Arts  degree  from  Ohio  Northern  University  and  a  master’s  degree  from 
Central  Michigan  University.  He  is  also  a  1998  graduate  of  the  Industrial  College  of  the  Armed  Forces, 
where  he  earned  a  second  masters  degree  in  national  resource  management.  In  addition,  he  is  a 
graduate  of  the  Defense  Systems  Management  College  at  Fort  Belvoir,  Senior  Executive  Fellows 
Program  from  the  Kennedy  School  of  Government  at  Harvard  University,  and  Leadership  for  a 
Democratic  Society  from  the  Federal  Executive  Institute. 

Mr.  Hazlett  was  recently  honored  with  the  Department  of  the  Air  Force’s  Award  for  Meritorious  Civilian 
Service.  He  is  married  and  has  two  children. 
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Enterprise  Challenges  Facing  the  Strategic  Sourcing  of  Services 


Presenter:  Stuart  Hazlett,  Deputy  Director  for  Strategic  Sourcing,  Defense  Procurement  and  Acquisition 
Policy 


The  Commander  of  Naval  Installations  and  the  Acquisition  of 
Services 


Presenter:  Thomas  Trump,  Assistant  Deputy  Commander  for  Contracting  Management,  Naval  Supply 
Systems  Command 


Performing  Quality  Assurance  and  Effectively  Administering  the 
Strategic  Sourcing  of  Services 


Presenter:  COL  Anthony  Incorvati,  Chief,  BTA  Division  and  Strategic  Sourcing,  ITEC4,  Army 
Contracting  Agency 

Ebbuhticil  Elements  to  Successful  Execution  of  Strategically 
Sourced  Services 


Presenter:  Randall  McFadden,  Director,  Acquisition  Management  and  Integration  Center 

Delivering  to  the  Warfighter 


Presenter:  Dale  Huegen,  Deputy  Chief,  Command  Acquisition,  USTRANSCOM 
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Introduction 


The  services  acquisition  volume  in  the  US  Department  of  Defense  (DoD)  has  continued 
to  increase  in  scope  and  dollars  in  the  past  decade.  In  fact,  in  recent  years,  the  DoD  has  spent 
more  on  services  than  on  supplies,  equipment  and  goods,  even  considering  the  high  value  of 
weapon  systems  and  large  military  items  (Camm,  Blickstein  &  Venzor,  2004).  Between  FY 
1999  to  FY  2003,  the  DoD’s  spending  on  services  increased  by  66%;  and  in  FY  2003,  the  DoD 
spent  over  $118  billion  (or  approximately  57%  of  total  DoD  procurement  dollars)  on  services 
(GAO,  2005a).  The  acquired  services  presently  cover  a  very  broad  set  of  service  activities, 
including:  professional,  administrative,  and  management  support;  construction,  repair,  and 
maintenance  of  facilities  and  equipment;  information  technology;  research  and  development, 
and  medical  care. 

As  the  DoD’s  services  acquisition  volume  continues  to  increase  in  scope  and  dollars,  the 
agency  must  give  greater  attention  to  proper  acquisition  planning,  adequate  requirements 
definition,  sufficient  price  evaluation,  and  proper  contractor  oversight  (GAO,  2002). 

In  our  previous  exploratory  research  on  the  challenges  and  opportunities  in  service 
supply  chains  in  the  DoD  (Apte,  Ferrer,  Lewis  &  Rendon,  2006),  we  reached  the  following 
preliminary,  yet  significant  conclusions: 

■  A  continued  growth  in  the  volume  and  scope  of  service  acquisition  in  the  DoD  in  the 
future 

■  As  observed  at  the  Presidio  of  Monterey,  innovative  supply-chain  arrangement  involving 
synergistic  contractual  relations  with  cities  adjacent  to  bases  for  the  management  of 
routine  municipal  services  can  result  in  significant  cost  savings. 

■  As  observed  at  Travis  AFB,  settings  conducive  to  successful  service  contracts  include: 
(1)  proactive  and  frequent  communications  among  cross-functional  teams  composed  of 
contracting  personnel  and  personnel  from  the  functional  organizations  involved  as 
customers  in  the  services  contracts,  and  (2)  co-location  of  contracting  professional  with 
the  customers  of  services. 

■  Acquisition  of  services  is  more  challenging  in  comparison  with  product/system 
acquisition  since  it  is  generally  more  difficult  to  establish  service  specifications  and 
measure  and  monitor  service  output  and  quality.  Hence,  having  onboard  the  right 
number  of  skilled  acquisition  personnel  is  highly  critical.  However,  the  observed 
downsizing  of  contracting  workforce  does  not  appear  to  be  in  line  with  this  need. 

■  Although  the  DoD  spends  more  on  acquiring  services  than  goods,  the  program 
management  infrastructure  for  the  acquisition  of  services  is  less  developed  than  that  for 
the  acquisition  of  products  and  systems.  In  many  service  acquisition  programs,  a 
trained  and  dedicated  program  manager  and  programs  management  team  does  not 
exist,  and  the  services  contracting  officer  becomes  the  de-facto  program  manager. 

The  lack  of  a  developed  program  management  infrastructure  for  the  acquisition  of 
services  is  a  critical  research  finding  that  warrants  further  study.  Review  of  the  current  literature 
shows  that  the  use  of  a  well-defined,  disciplined  approach  and  infrastructure  for  the 
management  of  projects  is  critical  for  a  project’s  success  in  meeting  cost,  schedule,  and 
performance  objectives  (Kerzner,  2006).  In  the  absence  of  a  well-defined  management 
infrastructure,  project  teams  are  left  to  create  an  ad  hoc  approach  to  managing  the  project. 
Based  on  our  exploratory  research,  we  believe  that  this  is  the  current  situation  in  many  DoD 
services  acquisition  programs.  The  lack  of  a  well-defined  program  management  infrastructure 
and  the  lack  of  a  lifecycle  approach  to  the  management  of  services  acquisition  projects  is 
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putting  the  success  of  these  critical  services  at  risk.  The  risks  for  not  meeting  cost,  schedule, 
and  performance  objectives  are  consequently  higher  in  the  DoD  services  acquisition  sector.  As 
the  DoD  increases  its  acquisition  of  services,  particularly  in  light  of  anticipated  budget  cuts  and 
dwindling  resources,  the  DoD  must  ensure  that  its  service  acquisition  projects  are  effectively 
and  efficiently  managed. 

The  purpose  of  this  research  is  to  continue  our  exploration  in  the  area  of  services 
contracts  while  focusing  on  the  implications  of  applying  a  program  management  structure  to 
services  acquisition.  We  will  first  discuss  some  continuing  issues  in  services  acquisition  based 
on  our  initial  research  and  recent  GAO  reports.  We  will  also  discuss  the  uniqueness  of  services 
and  how  they  affect  the  services  acquisition  process.  Next,  we  will  discuss  some  basic 
concepts  of  program  management  and  then  discuss  how  these  concepts  are  currently  being 
used  in  the  acquisition  of  systems  and  products,  specifically  defense  weapon  systems.  We  will 
then  discuss  the  application  of  program  management  and  project  management  concepts  to 
services  acquisition;  and  finally,  we  will  illustrate  how  program  management  concepts  can  be 
effectively  applied  at  the  various  levels  of  the  DoD  to  successfully  manage  service  acquisition 
programs. 

Continuing  Issues  in  Services  Acquisition 

a.  Conclusions  from  the  Prior  Research 

As  mentioned  before,  last  year  we  conducted  an  exploratory  research  in  Service  Supply 
Chain  in  the  Department  of  Defense.  The  conclusions  of  that  research  are  stated  below.  We 
want  to  clarify  that  given  the  exploratory  nature  of  that  research,  the  observations  and 
conclusions  listed  below  are  somewhat  preliminary  in  nature,  and  should  be  viewed  as  such. 

1 .  The  Department  of  Defense’s  services  acquisition  volume  has  continued  to  increase  in 
scope  and  dollars  in  the  past  decade.  The  GAO  (2005)  found  that  since  FY  1999,  the 
DoD’s  spending  on  services  has  increased  by  66%;  and  in  FY  2003,  the  DoD  spent  over 
$118  billion  (or  approximately  57%  of  total  DoD’s  procurement  dollars)  on  services.  DoD 
procures  a  variety  of  services,  including  both  the  traditional  commercial  service  and 
services  unique  to  defense.  In  terms  of  amount  spent,  four  service  categories  represent 
over  50%  of  total  spending  on  services:  (a)  professional,  administrative,  and 
management  support  services,  (b)  construction,  repair  and  maintenance  of  structure  and 
facilities,  (c)  equipment  maintenance,  and  (d)  information  technology  services. 

2.  Presidio  of  Monterey  (POM)  has  contracted  maintenance  of  about  155  buildings  and 
structures  to  Presidio  Municipal  Services  Agency  (PMSA),  a  consortium  of  the  cities  of 
Monterey  and  Seaside.  The  PMSA  agreement  has  allowed  the  two  cities  to  apply  their 
expertise  to  routine  municipal  services  and  the  Army  to  focus  on  its  military  mission. 
Through  this  partnership  and  contract  with  PMSA,  the  POM  has  realized  a  41% 
reduction  in  expenses  when  compared  with  previous  base  operation  costs  and  private 
contracts.  We  recommend  the  DoD  explore  and  evaluate  the  possibility  of  establishing 
such  synergistic  contractual  relations  with  cities  adjacent  to  other  bases  for  support  of 
their  respective  operations. 

3.  Proactive  and  frequent  communications  are  essential  for  a  successful  services  contract. 
We  found  a  successful  example  of  this  at  Travis  AFB,  where  60th  CONS  uses  BRAGs  as 
the  mechanism  for  conducting  such  communications.  Business  Requirement  Advisory 
Groups  (BRAGs)  are  teams  made  up  of  cross-functional  personnel  that  represent  the 
functional  organizations  involved  as  customers  in  the  services  contracts.  These  cross¬ 
functional  teams  plan  and  manage  the  service  contracts  throughout  the  service’s 
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lifecycle.  As  the  DoD  increases  the  use  of  centralized  contracting  organizations  and 
regional  contracts,  the  use  of  proactive  and  frequent  communications  will  be  even  more 
essential  for  the  successful  management  and  performance  of  these  contracts. 

4.  Our  visits  and  interviews  at  Travis  AFB,  Presidio  of  Monterey  (POM),  Naval  Air  Station 
Whidbey  Island  (NAS  Wl),  and  the  Naval  Support  Detachment  Monterey  (NSDM) 
confirmed  the  GAO’s  finding  that,  “while  the  Army’s  and  Navy’s  creation  of  centralized 
installation  management  agencies  can  potentially  create  efficiencies  and  improve  the 
management  of  the  facilities  through  streamlining  and  consolidation,  implementation  of 
these  plans  has  so  far  met  with  mixed  results  in  quality  and  level  of  support  provided  to 
activities  and  installations”  (GAO,  2005b). 

5.  The  centralization  of  contracting  offices  and  use  of  regional  contracts  will  result  in 
additional  dynamics  to  the  DoD’s  acquisition  of  services.  The  use  of  centralized 
contracting  organizations  and  regional  contracts  will  require  even  more  proactive  and 
frequent  communications  between  the  contracting  organization  and  the  customer. 
Although  it  is  still  too  early  to  assess  the  effectiveness  and  efficiency  of  centralized 
contracting  organizations  and  regional  contracts,  this  research  has  indicated  that 
centralization  and  regionalization  of  services  contracts  are  growing  trends  in  the  DoD 
and  will  significantly  change  how  services  contracts  are  managed. 

6.  Given  the  unique  characteristics  of  services  (such  as  intangibility,  co-production, 
diversity  and  complexity),  establishing  service  specifications  and  measuring  and 
monitoring  the  quality  of  delivered  service  is  inherently  more  complex  than  when  dealing 
with  manufactured  goods.  Hence,  it  is  critical  to  have  onboard  a  “knowledgeable  client” 
and  the  necessary  number  of  skilled  contracting  personnel  to  define  the  requirements 
and  to  supervise  outsourced  services.  The  DoD  has  been  aggressively  complying  with 
OMB’s  Circular  A-76,  which  directs  all  federal  government  agencies  “to  rely  on  the 
private  sector  for  needed  commercial  activities.”  This  has  resulted  in  dramatic  growth  in 
the  DoD’s  spending  and  in  the  downsizing  of  the  DoD  civilian  and  military  acquisition 
workforce.  Although  this  exploratory  study  is  not  yet  completed,  we  believe  that  the 
above  two  trends  contradict  the  critical  need  to  have  onboard  a  necessary  number  of 
skilled  contracting  personnel.  This  could  mean  that  in  the  DoD’s  outsourced  services, 
either  the  needs  are  not  being  fully  satisfied,  or  the  value  for  the  money  spent  is  not 
being  realized. 

7.  Although  the  DoD  acquires  more  services  than  goods,  and  the  acquisition  of  services 
and  the  use  of  service  contractors  are  becoming  an  increasingly  critical  aspect  of  the 
DoD  mission,  the  management  infrastructure  for  the  acquisition  of  services  is  less 
developed  than  for  the  acquisition  of  products  and  systems.  There  is  a  less  formal 
program  management  approach  and  lifecycle  methodology  for  the  acquisition  of 
services,  which  is  confirmed  by  the  lack  of  standardization  in  the  business  practices 
associated  with  the  services  acquisition  process.  This  results  from  the  fact  that  the 
functional  personnel  currently  managing  the  services  programs  are  not  considered 
members  of  the  DoD  acquisition  workforce,  and  are  typically  not  provided  acquisition 
training  under  Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  requirements. 

b.  Service  Characteristics  and  Their  Implications  to  Contracting 

Intangibility  of  service  outcomes  makes  it  difficult  to  clearly  describe  and  quantify  services, 
and,  therefore,  to  contract  for  services.  Consider,  for  example,  the  difficulty  in  writing  a  contact  for 
an  educational  service  involving  academic  lectures.  How  does  one  define  a  “pound  of  education,” 
and  how  can  one  be  sure  when  the  contract  is  fulfilled  satisfactorily.  As  Karmarkar  and  Pitbladdo 
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(1993)  explain,  this  is  the  reason  why  in  such  cases  we  do  not  contract  around  quantities  at  all; 
rather,  we  contract  around  process  delivery.  In  general,  the  more  information  intensive  the 
service  is,  the  more  difficult  it  is  to  develop  clear  and  meaningful  contracts.  This  difficulty  is 
somewhat  reduced  in  services  in  which  physical  objects  play  a  dominant  role. 

Intangibility  of  outputs  also  makes  it  difficult  to  define  and  measure  quality.  For  example, 
even  for  a  simple  custodial  service  such  as  cleaning,  it  is  not  easy  to  define  the  desired  level  of 
cleanliness.  The  levels  of  cleaning  needed  for  an  office  is  certainly  different  than  for  a  hospital 
operating  room.  The  desired  time  duration  for  maintaining  a  clean  status  can  also  be  an 
important  matter  in  writing  a  contract  for  cleaning  service.  As  research  in  service  quality  has 
found,  customers  typically  evaluate  the  quality  of  service  based  on  the  outcome  of  a  service  as  well 
as  the  customer’s  experience  with  the  process  of  service  delivery.  For  example,  in  a  dining  facility, 
not  only  must  the  food  be  tasty,  but  the  manner  in  which  the  food  is  served  must  also  be  courteous, 
prompt  and  friendly.  This  means  that  the  contracts  for  many  services  should  not  be  based  solely 
on  outcomes  but  should  include  specifications  on  both  the  outcome  and  the  customer’s  experience 
with  the  process. 

Co-production  requiring  presence  and  participation  of  customers  in  the  creation  of  many 
services  is  an  important  characteristic  of  services.  For  example,  in  an  IT  services  such  as  software 
development,  a  customer’s  input  in  terms  of  desired  specifications  of  a  software  system  is  critically 
important.  For  example,  however  competent  the  software  developer  may  be,  the  developed 
software  will  not  be  satisfactory  if  the  specifications  do  not  accurately  reflect  the  true  needs  of  the 
customer.  Hence,  the  contracts  for  services  should  ideally  specify  not  only  what  the  service 
provider  should  do  but  also  what  the  customer  should  do.  Otherwise,  a  satisfactory  service 
outcome  may  not  be  realized. 

Diversity  of  Services  also  makes  it  difficult  and  undesirable  to  use  the  same  contract 
vehicles  or  procedures  for  different  services.  For  example,  given  the  differences  in  medical 
services  versus  custodial  services,  it  is  important  that  the  contracts  for  these  services  are 
customized  to  suit  the  lifecycle  needs  of  individual  services. 

Finally,  services  are  complex  and  may  involve  multi-stage  processes.  This  makes  it 
important,  yet  challenging,  to  write  contracts  that  are  flexible  enough  to  cover  all  relevant 
scenarios  and  eventualities.  Moreover,  if  such  contracts  cannot  be  satisfactorily  defined,  it  may 
be  more  desirable  to  deliver  certain  services  using  internal  resources  than  to  outsource  them. 

c.  Service  Markets  and  Contracts 

The  above-discussed  special  features  of  services  lead  to  significant  differences  in  the 
process  of  production,  sale  and  consumption  of  services.  These,  in  turn,  have  implications  for 
market  structure,  pricing,  and  contracting  for  services.  While  the  operational  implications  of  service 
characteristics  have  received  some  attention,  there  have  been  very  few  attempts  to  capture  the 
implications  for  markets.  The  large  majority  of  papers  dealing  with  service  competition  have 
addressed  issues  like  queues  and  congestion,  and  their  consequences  for  customer  waiting  time. 

While  queuing  is  certainly  an  issue  central  to  services — customers  must  access  service 
systems  because  of  the  lack  of  portability  of  services — the  difference  relative  to  manufacturing  is 
primarily  one  of  degree.  There  are,  on  the  other  hand,  several  important  characteristics  of  services 
which  remain  untreated  in  terms  of  market  models.  For  example,  there  is  little  to  be  found  on  the 
subject  of  models  with  joint  production.  Similarly,  the  inability  to  measure  and  meter  service  output 
renders  standard  price-quantity  mechanisms  untenable.  The  result  is  that  prices  must  be  set  on  a 
case  basis,  by  specific  bilateral  contracting  based  on  inputs  rather  than  outputs,  or  by  repeated 
renegotiation  and  contracting.  While  these  are  not  individually  all  new  issues,  there  does  not  seem 
to  be  an  integrated  treatment  of  service  markets  from  this  viewpoint. 
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Karmarkar  and  Pitbladdo  (1993)  present  some  key  features  regarding  service  contracting 
that  are  relevant  to  the  development  of  a  service  quality  model.  First  and  foremost,  service 
operations  are  always  post-contractual  (with  the  possible  exception  of  New  York  City  automotive 
window  washing).  Fixed-price  contracts  centered  on  output  specifications  can  fail  on  two  accounts. 
First  is  the  difficulty  of  conceiving  or  verifying  meaningful  output  specifications,  and  second  is  the 
variability  of  customer  inputs  and  joint  production  which  makes  fixed-price  contracts  risky  for  the 
firm  even  when  the  output  specifications  can  be  well  defined.  Alternatively,  contracts  based  on 
process  specifications,  such  as  time  and  materials,  can  turn  out  to  be  unsuitable  since  these  can 
be  risky  for  customers.  These  dual  risks  for  firms  and  for  customers  can  be  addressed  via  stage- 
wise  or  contingent  contracting,  in  which  the  process  is  broken  into  stages,  and  the  price  for  a  given 
stage  is  made  dependent  on  the  outputs  of  previous  stages.  For  example,  there  may  be  a  fixed 
fee  for  a  diagnosis,  and  a  fixed  fee  for  treatment  which,  however,  depends  on  the  outcome  of  the 
diagnosis.  The  uncertainty  in  customer  inputs  is  resolved  by  the  diagnosis  before  it  materializes  in 
terms  of  treatment  cost. 

d.  Stage-wise  Decomposition  of  Services 

The  presence  of  a  tangible,  portable  output  which  can  be  quantified  by  both  vendor  and  buyer 
allows,  perhaps  forces,  considerable  simplification  in  the  market-forming  process  of  manufactured 
goods.  Contracts  for  manufactured  goods  are  centered  around  a  clearly  defined  junction  between 
production  and  use,  at  which  point  responsibility  is  transferred  from  producer  to  customer.  While 
the  value  of  a  product  to  a  customer  may  actually  depend  on  the  customer-specific  uses  to  which 
the  product  is  put,  such  information  is  not  needed  at  the  market  interface,  where  customers  can 
reveal  their  preferences  through  price-quantity  negotiations.  Similarly,  specifications  of  the 
production  process  have  no  relevance  at  the  market  interface  apart  from  their  impact  on  the 
specifications  of  the  product. 

For  services,  the  transaction  between  customer  and  provider  must  be  represented  in 
greater  detail.  Figure  1  shows  the  sequence  of  steps  involved  in  a  service  transaction  as  seen  by 
a  customer.  At  the  end  of  each  step  is  the  state  that  is  reached,  observed  by  either  the  buyer  or 
the  vendor  of  the  service.  Karmarkar  and  Pitbladdo  discuss  why,  1 )  contract  terms  for  the  next 
stage  are  typically  contingent  on  the  states  reached  in  the  previous  stages,  and  2)  switching  to 
competing  providers  is  an  option  at  the  end  of  each  stage.  We  hasten  to  note  that  not  all  services 
necessarily  involve  all  these  steps. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-  587- 


Figure  1.  A  Conceptual  Model  of  a  Service  Lifecycle 

The  first  step  is  access  to  the  service;  this  may  involve  bringing  a  customer  to  the  service 
system,  or  the  reverse.  The  second  stage  is  diagnosis.  For  our  purposes,  diagnosis  is  defined  as 
the  mapping  of  customer  requirements  of  a  service  into  a  technical  or  process  specification.  For 
example,  a  customer  at  a  car  repair  facility  may  describe  certain  problems  that  he  or  she  has 
experienced  based  on  which  repair  needs  can  be  assessed.  The  process  of  diagnosis  identifies 
the  underlying  technical  problem,  perhaps  as  a  syndrome  (collection  of  symptoms)  or  perhaps  in 
terms  of  the  underlying  mechanism.  The  medical  analogy  is  obvious.  Similarly,  a  client  of  a 
financial  planning  service  may  describe  problems  in  terms  of  college  payments  and  retirement. 

The  planner  may  convert  the  statement  to  needed  cash  flows,  risk  attitudes  and  state  preferences. 
Diagnosis  is  likely  to  be  an  interactive  or  joint  process.  In  some  cases,  the  diagnosis  step  is 
performed  by  the  customer  alone. 

The  third  stage,  process  planning,  is  the  generation  of  alternative  service  processes  or 
treatments  to  meet  the  output  requirements  defined  by  the  diagnosis.  This  may  be  a  joint 
production  process.  In  some  cases,  the  alternatives  available  are  already  stated  and  fixed.  A 
menu  at  a  restaurant  is  an  example.  In  others,  processes  or  alternatives  can  be  highly  specialized 
to  the  customer's  needs.  It  is  conceptually  useful  to  note  that  diagnosis,  coupled  with  process 
planning,  is  the  dynamic  equivalent  of  Quality  Function  Deployment  (QFD)  (Hauser  &  Clausing, 
1988).  In  manufacturing,  QFD  consists  of  mapping  generic  customer  needs  into  clearly  defined 
product  specifications.  Diagnosis  and  process-plan  generation  consist  of  mapping  specific 
customer  needs  and  desires  into  clearly  defined  process  specifications,  particular  to  the  customer. 

The  fourth  stage  is  the  execution  of  the  service  process  itself.  Once  again,  this  may  or  may 
not  involve  joint  production.  Finally  we  add  a  fifth  stage,  continuation,  which  represents  the 
continuing  consumption  or  consequences  of  service  outputs  (the  provider's  role  in  this  stage  can 
be  characterized  as  long-term  service  support).  The  reason  for  this  is  that  the  outputs  or 
consequences  of  many  services  (e.g.,  health  care,  financial  planning,  consulting)  cannot  be 
completely  evaluated  immediately.  It  is  instructive  to  note  here  that,  in  the  manufacturing  case,  the 
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counterpart  of  this  fifth  stage  constitutes  the  entirety  of  the  customer  involvement  with  the  product. 
The  service  provider  may  continue  to  have  a  role  in  this  stage  in  the  form  of  direct  interaction  and 
consultation  or  a  set  of  instructions  along  the  lines  of  a  "user's  manual."  Surgery  provides  a  clear 
example,  involving  a  schedule  of  required  and  proscribed  activities,  along  with  follow-up  checkups 
and  telephone  consultations. 

In  the  next  section,  we  build  on  some  concepts  discussed  above  to  propose  a  program 
management  approach  for  services  acquisition. 

Towards  a  Program  Management  Approach  to  Services 
Acquisition 

This  research  on  the  acquisition  of  services  will  focus  on  the  application  of  a  program- 
management  approach  and  project-management  concepts  to  services  acquisition.  This  section 
will  first  discuss  some  basic  concepts  of  program  management  and  then  discuss  how  these 
concepts  are  currently  being  used  in  the  acquisition  of  systems  and  products,  specifically 
defense  weapon  systems.  The  next  section  will  discuss  the  application  of  a  program- 
management  approach  and  project-management  concepts  to  services  acquisition. 

a.  An  Overview  of  the  Program  Management  Approach 

Review  of  the  current  literature  shows  that  the  use  of  a  well-defined,  disciplined 
methodology  and  infrastructure  for  the  management  of  complex  projects  is  critical  for  a  project’s 
success  in  meeting  cost,  schedule,  and  performance  objectives  (PMI,  2004,  Kerzner,  2006).  We 
use  the  term  “program  management”  to  describe  the  approach  and  methodology  needed  for  the 
management  of  complex  projects.  A  program  management  approach  includes  the  infrastructure 
that  facilitates  the  successful  attainment  of  cost,  schedule,  and  performance  objectives.  A 
program  management  approach  refers  to  the  centralized,  coordinated  management  of  a  group 
of  projects  to  achieve  the  program’s  strategic  objectives  and  benefits  (PMI,  2004).  In  addition, 
programs  themselves  consist  of  related  projects  managed  in  a  coordinated  way  to  obtain 
benefits  and  control  (PMI,  2004).  Thus,  a  disciplined  program  management  approach  includes 
the  following  project  management  concepts:  project  lifecycle,  integrated  project  processes,  an 
assigned  and  dedicated  project  manager,  empowered  cross-functional  project  teams,  and  an 
appropriate  project  organizational  structure.  These  project  management  concepts  will  be  briefly 
discussed. 

1.  Project  Lifecycle 

An  effective  way  of  managing  projects  is  to  divide  the  project  into  phases  to  provide 
better  management  and  control.  These  phases  make  up  the  project  lifecycle.  The  phases  of 
the  project  lifecycle  can  be  used  to  manage  and  control  the  activities  that  are  conducted  within 
each  project  phase.  By  using  the  phases  of  the  project  lifecycle  and  establishing  control  gates 
or  milestones  between  project  phases,  the  project  manager  can  control  the  progression  of  the 
project.  Although  project  lifecycles  are  different  for  each  specific  type  of  project,  many 
organizations  will  establish  a  standardized  lifecycle  for  their  projects.  Typically,  the  project 
lifecycle  of  a  system  consist  of  the  following  phases:  conceptual,  planning,  testing, 
implementation,  and  closure  (Kerzner,  2006).  Later  in  this  report,  we  will  discuss  how  the 
project  lifecycle  is  used  in  defense  weapon  system  projects.  We  will  also  discuss  the 
development  of  a  project  lifecycle  for  service  acquisition  projects. 
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2.  Integrated  Processes 

A  disciplined  program  management  approach  includes  the  integration  of  various  project 
management  processes  used  throughout  the  project.  These  various  project  processes  typically 
include  such  processes  as  requirements  development,  scheduling,  cost  management,  quality 
management,  risk  management,  and  contracts  management  (PMI,  2004).  Although  each  of 
these  specific  project  processes  reflects  different  functional  areas,  a  disciplined  program 
management  methodology  would  integrate  these  various  processes  to  ensure  that  each  of 
these  areas  are  coordinated  and  integrated  within  the  total  project  effort.  In  addition  to 
integrated  processes,  the  PMI  project  management  body  of  knowledge  establishes  five  project 
management  process  groups.  These  project  management  process  groups  include  initiating 
processes,  planning  processes,  monitoring  and  controlling  processes,  and  closing  processes 
(PMI,  2004).  Each  of  these  project  management  process  groups  includes  various  functional 
processes  that  are  part  of  that  specific  process  group.  For  example,  the  planning-process 
group  would  include  such  processes  as  scope  planning,  quality  planning,  risk-management 
planning,  procurement  planning  and  solicitation  planning.  In  order  to  effectively  and 
successfully  manage  projects,  these  various  functional  processes  need  to  be  coordinated  and 
integrated  throughout  the  total  project  effort.  Having  integrated  project  processes  are  key  to 
successful  project  management. 

3.  Project  Teams 

Just  as  integrated  processes  are  an  essential  for  effective  project  management, 
integrated  project  teams  are  also  essential.  A  disciplined  program  management  methodology 
includes  the  establishment  of  integrated  project  teams  consisting  of  project  team  members 
representing  each  of  the  different  functional  areas  that  are  part  of  the  project  effort.  For 
example,  a  project  team  may  include  functional  experts  representing  the  various  processes  that 
are  used  in  the  project  such  as  risk  management,  requirements  management,  and  contracts 
management.  These  functional  experts  on  the  project  team  are  responsible  for  providing  their 
expertise  in  support  of  the  project  objective.  Although  the  project  team  consists  of  these  various 
functional  experts,  the  activities  of  these  project  team  members  must  be  coordinated  and 
integrated  to  ensure  accomplishment  of  the  project  object.  The  coordination  and  integration 
responsibility  belongs  to  the  project  manager. 

4.  Project  Manager 

A  critical  aspect  of  a  disciplined  project  management  methodology  includes  the 
assignment  of  a  dedicated  project  manager  to  oversee  the  activities  of  the  project.  We  have 
already  stated  that  the  project  effort  includes  various  functional  processes  conducted  by 
functional  experts  on  the  project  team.  We  also  stated  that  there  are  project  management 
process  groups  that  are  used  to  help  integrate  these  various  functional  processes.  The  role  of 
the  project  manager,  therefore,  is  to  coordinate  and  integrate  the  various  project  activities  to 
ensure  successful  completion  of  the  project  (Kerzner,  2006).  The  project  manager  is 
responsible  for  ensuring  that  all  members  of  the  project  team  support  the  projects  objectives. 
Thus,  having  a  dedicated  project  manager,  who  is  responsible  for  managing  the  project 
activities  and  ensuring  the  achievement  of  the  project  objectives,  is  an  essential  part  of  a 
disciplined  project  management  methodology. 
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5.  Organizational  Structure 

An  appropriate  organizational  structure  is  also  an  essential  element  of  a  disciplined 
project  management  methodology.  An  organizational  structure  that  supports  the  integrated 
project  management  processes,  integrated  project  teams,  and  the  roles  and  responsibilities  of 
the  project  manager  will  significantly  contribute  to  the  success  of  the  project.  One  of  the  three 
main  types  of  organizational  structures — functional,  matrix,  and  pure  project — is  typically  used  in 
organizations  that  perform  projects  (PMI,  2004).  The  degree  of  project  manager  authority, 
resource  availability  and  budget  control  will  be  affected  by  the  type  of  organizational  structure 
(PMI,  2004).  Some  of  the  factors  to  consider  in  selecting  the  appropriate  type  of  organizational 
structure  include:  the  number  of  functional  areas  involved  in  the  project,  the  level  of  integration 
needed  within  the  functional  areas  and  between  the  organization  and  the  customer,  the  nature 
of  the  technology  used  in  the  project  work,  and  the  organization’s  previous  experience  in 
performing  the  work  required  by  the  project. 

b.  Application  of  Program  Management  Concepts  to  Weapon  Systems  Acquisition 

The  previous  section  discussed  the  basic  project  management  concepts  such  as  the 
project  lifecycle,  integrated  processes,  project  teams,  project  manager,  and  organizational 
structure.  These  program  management  concepts  are  well  established  in  the  Department  of 
Defense  weapon  systems  acquisition  environment.  In  fact,  many  of  today's  modern  project 
management  tools  and  techniques  were  developed  during  the  Cold  War.  Weapon  system 
programs  such  as  the  land-based  ICBM  and  sea-based  ballistic  missile  programs  became  the 
proving  grounds  for  some  of  today's  modern  program  management  processes  (Kerzner,  2006). 
In  today's  DoD  weapon  systems  acquisition  environment,  program  management  concepts 
continue  to  be  integral  to  the  successful  management  of  these  critical  and  high-technology 
projects. 

The  Department  of  Defense  Directive  5000.1  establishes  the  defense  acquisition  system 
as  the  management  process  by  which  the  DoD  provides  effective,  affordable,  and  timely 
systems  to  the  users  (DoD,  2003).  This  directive  establishes  the  role  of  the  program  manager 
as  the  designated  individual  authorized  and  responsible  for  accomplishing  the  program 
objectives.  The  program  manager  is  the  designated  individual  that  is  accountable  for  costs 
schedule  and  performance  reporting  to  the  milestone  decision  authority  (MDA)  (DoD,  2003). 

The  Department  of  Defense  Instruction  5000.2  establishes  the  defense  acquisition 
management  framework  as  the  project  lifecycle  for  major  defense  acquisition  programs 
(USD(AT&L),  2003,  May  12).  This  lifecycle  consists  of  the  various  phases,  decision  points,  and 
project  review  points  that  are  part  of  the  project  lifecycle.  See  Figure  2  for  an  illustration  of  the 
DoD  Acquisition  Management  Framework. 
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Figure  2.  The  Defense  Acquisition  Management  Framework 

In  addition,  the  DoD  5000  regulations  also  established  the  use  of  integrated  product 
teams  (IPTs)  and  integrated  processes  throughout  the  weapon  systems  acquisition 
management  lifecycle.  Through  the  use  of  effective  collaboration,  program  managers  are 
responsible  for  making  project  decisions  and  leading  project  execution  by  maintaining 
continuous  and  effective  communication  through  use  of  integrated  project  processes. 

Finally,  for  weapon  systems  acquisition  management,  the  DoD  relies  heavily  on  unique 
organizational  structures  such  as  the  matrix  organizational  structure  and,  in  some  instances, 
project-type  structures  for  the  management  of  defense  acquisition  programs.  Figure  3  is  an 
example  of  an  organizational  structure  for  a  weapon  system  acquisition  program. 
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Figure  3.  Organization  Structure  of  a  Weapon  System  Acquisition  Program 
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Thus,  the  basic  concepts  reflective  of  a  program  management  approach  are  well 
established  in  DoD  weapon  systems  acquisition  management.  The  use  of  project  lifecycles  and 
control  gates,  integrated  processes,  established  project  manager  and  project  teams,  and  an 
effective  organizational  structure  have  been  successfully  used  in  this  specific  sector  of  the 
Department  of  Defense.  The  next  section  of  this  research  will  discuss  how  this  program 
management  approach  (consisting  of  these  basic  project  management  concepts)  can  be  used 
by  the  DoD  in  the  acquisition  of  services. 

Applications  of  Program  Management  Concepts  to  Services 
Acquisition 

Our  previous  discussion  focused  on  the  use  of  a  program  management  approach  and  its 
related  project  management  concepts  found  in  the  current  literature.  These  project 
management  concepts  included  the  use  of  a  project  lifecycle  and  control  gates,  integrated 
processes,  a  dedicated  program  manager  and  integrated  project  teams,  and  an  organizational 
structure  conducive  to  the  integration  of  project  activities.  We  also  discussed  how  this  program 
management  approach  and  these  project  management  concepts  have  been  well  established  in 
the  weapon  systems  acquisition  management  environment.  We  identified  Department  of 
Defense  directives  and  instructions  that  support,  even  mandate  the  use  of  some  of  these  project 
management  concepts.  This  section  will  now  focus  on  the  acquisition  of  services  within  the 
Department  of  Defense.  We  will  look  at  the  acquisition  of  services  at  the  installation  level, 
command  level,  and  service  level.  The  purpose  here  is  to  illustrate  how  a  program 
management  approach  and  project  management  concepts  can  be  effectively  applied  at  each  of 
these  levels  to  successfully  manage  service  acquisition  programs. 

In  exploring  DoD  services  acquisition,  our  research  included  basic  installation-level  types 
of  services  that  are  commonly  acquired  in  support  of  the  installation  mission.  The  installations 
researched  included  Travis  Air  Force  Base  California,  Randolph  Air  Force  Base  Texas,  Presidio 
of  Monterey  California,  and  the  Naval  Postgraduate  School,  California.  At  the  installations 
visited,  personnel  were  interviewed  for  this  research;  we  determined  that  although  some  project 
management  concepts  were  applied,  they  were  not  applied  in  a  consistent  manner,  or  were  not 
institutionalized  throughout  the  organization.  In  addition,  at  many  of  these  locations,  although 
project  management  tools  and  concepts  were  being  applied,  this  utilization  did  not  necessarily 
result  in  a  program  management  approach  to  acquiring  services. 

Typically,  at  the  installation  level,  the  acquisition  of  services  is  managed  using  more  of 
an  ad  hoc  approach  as  opposed  to  a  program  management  approach. 

1.  Project  Lifecycle 

In  terms  of  using  a  project  lifecycle,  our  research  indicated  that  the  contracting  process 
was  typically  used  as  the  project  lifecycle.  Although  the  contracting  process  is  an  integral  part 
of  the  acquisition  lifecycle  for  DoD  weapon  systems  acquisition,  the  project  lifecycle  is  separate 
from  the  contracting  process  lifecycle. 

2.  Integrated  Processes 

In  addition,  our  research  indicated  that  although  various  project  management  processes 
were  used  at  the  installation  level,  these  processes  were  not  necessarily  integrated  in  the 
management  of  the  services  contracts.  Although  we  did  find  the  various  project  management 
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processes  (such  as  contracting  process,  risk-management  process,  quality-assurance  process, 
and  contract-funding  process),  we  did  not  see  the  integration  of  these  processes. 

3.  Project  Teams 

Furthermore,  the  use  of  informal  project  teams  was  in  existence  at  these  installation- 
level  organizations.  Although  these  project  teams  were  in  existence,  the  structures  of  these 
project  teams  were  created  more  on  an  ad  hoc  basis  than  established  as  formal  project  teams. 

4.  Project  Manager 

Closely  related  to  the  above  was  the  ad  hoc  approach  to  the  establishment  of  a  project 
manager  for  services  acquisition.  In  many  cases,  the  project  manager,  or  program  manager,  as 
sometimes  called,  existed  at  the  major  command  headquarters  level  as  opposed  to  the 
installation  level.  As  we  will  discuss  in  the  next  section  on  Major  Command  level  management 
of  services  acquisition,  we  found  that  many  service  acquisitions  were  centrally  planned  at  the 
headquarters  and  then  de-centrally  executed  at  the  installation  level.  If  there  is  an  assigned 
program  manager,  that  individual  is  typically  assigned  at  the  Major  Command  level,  with 
oversight  responsibility  for  the  installation-level  activity.  In  these  instances,  although  the 
program  manager  was  located  at  the  headquarters  level,  there  was  no  program  manager  at  the 
installation  level.  In  this  case,  although  the  quality  assurance  evaluators  (QAE)  represented  the 
program  manager,  the  QAE  does  not  perform  program  management  responsibilities.  In 
addition,  the  contracting  officer  at  the  installation  typically  functioned  as  the  de-facto  program 
manager  due  to  the  lack  of  any  program  manager  at  the  site.  It  should  be  noted  that  the 
procuring  contracting  officer  (PCO)  at  the  Major  Command  headquarters  where  the  services 
acquisition  was  centrally  planned  and  executed  delegated  the  contract  to  the  administrative 
contracting  officer  at  the  installation  where  the  contract  was  administered.  Thus,  the  PCO, 
responsible  for  proving  contracting  support  for  the  centrally  planned  and  executed  services 
acquisition,  would  delegate  the  administration  tasks  to  the  ACO  for  the  decentralized 
administration  of  the  services  contract.  However,  the  program  manager  retained  the  program 
management  functions  of  the  services  acquisition. 

5.  Organizational  Structure 

Finally,  in  terms  of  organizational  structures,  at  the  installation  level  our  research  did  not 
identify  any  specific  or  unique  organizational  structures  specifically  established  for  the 
acquisition  of  services.  The  installations  we  researched  reflected  the  traditional  organizational 
structures  and  organizational  mission  of  the  Defense  Department.  We  did  not  see  any 
projectized  or  matrix  organizational  structures  used  in  the  management  of  services  contracts  at 
the  installation  level. 

Preliminary  Conclusions 

Our  preliminary  conclusion  in  this  ongoing  exploratory  research  has  identified  some 
unique  aspects  of  services  and  how  they  affect  the  services  acquisition  process.  We  developed 
a  conceptual  model  of  a  service  lifecycle  that  can  be  used  in  analyzing  the  DoD’s  services 
acquisition  process.  We  also  discussed  the  program  management  approach  and  identified 
basic  project  management  concepts  and  discussed  how  these  concepts  are  being  used  in  the 
acquisition  of  defense  weapon  systems.  Our  current  research  has  observed  that  the  program 
management  approach  is  applicable  to  the  acquisition  of  services  within  the  DoD.  We  have 
also  initially  concluded  that  the  basic  project  management  concepts  (such  as  project  lifecycle, 
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integrated  processes,  project  team,  project  manager,  and  organizational  structure)  can  be 
applied  to  the  acquisition  of  services.  Our  currents  research  leads  us  to  believe  that  the 
application  of  a  program  management  approach  and  the  adoption  of  basic  project  management 
concepts  to  the  acquisition  of  services  will  improve  the  management  and  oversight  of  these 
services  contracts.  Our  further  research  will  explore  in  more  detail  how  this  program 
management  approach  and  project  management  concepts  can  be  used  to  improve  the  DoD’s 
services  acquisition  management. 
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Panel  20  -  Military  Logistics  and  Maintenance:  Beyond  Lean  and  Six 
Sigma 


Thursday, 
May  17,  2007 


3:30  p.m.  -  Chair: 
5:00  p.m. 


Garry  B.  Richey,  Executive  Director,  Oklahoma  City  Air  Logistics  Center, 
Tinker  Air  Force  Base 

Discussant: 

Dale  L.  Moore,  Deputy  Corporate  Deployment  Champion  NAVAIR 
AIRSpeed. 


Papers: 


Lean  Six  Sigma  Implementation  for  Military  Logistics  to  Improve 
Readiness:  Enterprise  Challenges  Facing  the  Strategic  Sourcing  of 
Services 

Keebom  Kang  and  Uday  Apte,  Naval  Postgraduate  School 

Beyond  AirSpeed:  How  Organizational  Modeling  and  Simulation  Further 
Reduced  Engine  Maintenance  Time 

Lt.  Col.  Joel  J.  Hagan,  USAF,  Capt.  William  G.  Slack,  USMC,  Roxanne 
Zolin  and  John  Dillard,  Naval  Postgraduate  School 


Chair:  Garry  B.  Richey,  Executive  Director,  Oklahoma  City  Air  Logistics  Center,  Tinker  Air  Force  Base, 
was  selected  for  reassignment  as  Director,  Logistics,  Installations  and  Mission  Support,  Headquarters  Air 
Education  and  Training  Command,  Randolph  Air  Force  Base,  Texas. 

Garry  B.  Richey,  a  member  of  the  Senior  Executive  Service,  is  Executive  Director,  Oklahoma  City  Air 
Logistics  Center,  Tinker  Air  Force  Base,  Okla.  He  is  responsible  for  assisting  the  commander  in  providing 
worldwide  logistics  support  for  assigned  Air  Force  weapon  systems,  including  the  C/KC-135,  E-3,  B-1B, 
B-52  and  the  B-2  stealth  bomber. 

Mr.  Richey  was  born  in  Chickasha,  Okla.,  and  graduated  from  Duncan  (Okla.)  High  School.  He  was 
drafted  into  the  U.S.  Army  in  1972  and  served  at  Fort  Benning,  Ga.,  until  he  was  discharged  in  1974.  In 
1 977,  he  received  a  bachelor's  degree  in  speech  communication  from  the  University  of  Oklahoma  and 
started  his  civil  service  career  as  a  logistics  management  specialist  at  Tinker  AFB,  Okla.  His  career 
includes  acquisition  and  logistics  experience  as  a  director  and  logistics  manager  in  propulsion,  aircraft 
and  item  management.  Mr.  Richey  also  served  as  the  Proposal  Manager  for  the  Tinker  AFB  bid  in  the 
Kelly  AFB  Propulsion  Business  Area  public-private  competition,  which  was  awarded  to  Tinker  AFB  in 
February  1999. 

Prior  to  his  current  position,  Mr.  Richey  was  the  Deputy  Director  of  Logistics,  Headquarters  Air  Mobility 
Command,  Scott  AFB,  III.  He  was  responsible  for  providing  policy  and  guidance  to  organize,  train  and 
equip  aircraft  maintenance,  transportation,  supply,  contracting,  and  logistics  plans  units  at  14  major  active 
air  installations  in  the  United  States,  and  17  en  route  locations  around  the  world.  He  also  assisted  in  the 
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readiness  of  more  than  616  aircraft  and  80,339  people  in  the  Air  Force  Reserve  and  Air  National  Guard, 
providing  total  force  augmentation  to  support  flexible,  global  reach  for  America. 


Discussant:  Dale  L.  Moore  has  served  as  the  NAVAIR  AIRSpeed  Deputy  Corporate  Deployment 
Champion  in  support  of  RDML  Mark  Skinner,  the  NAE  AIRSpeed  Deployment  Champion  since  May  2004. 
NAVAIR  AIRSpeed  supports  the  command’s  deployment  of  continuous  process  improvement  and 
productivity-related  strategies,  tools,  and  methodologies,  including  Lean,  Six  Sigma,  and  Theory  of 
Constraints.  NAVAIR  AIRSpeed  deployment  is  designed  to  drive  process  efficiencies  and  embed  a 
culture  of  productivity  throughout  an  enterprise  of  more  than  37,000  military,  civilian,  and  contractor 
support  personnel.  Moore  leads  the  Naval  Aviation  Enterprise  AIRSpeed  Merge  Governance  Team  to 
develop  the  organizational  governance  construct  for  deploying  and  executing  AIRSpeed  across  the 
180,000  person  NAE  workforce. 

Moore  is  a  trained  George  Group  Lean  Six  Sigma  Black  Belt  and  holds  a  master’s  degree  in  product 
development  (with  distinction)  from  the  Navy  Postgraduate  School  with  certificates  of  completion  from  the 
M.l.T.  Sloan  School  of  Management  and  the  M.l.T.  School  of  Engineering,  a  bachelor’s  degree  in 
mechanical  engineering  from  the  University  of  Delaware,  and  is  a  graduate  of  the  Defense  Systems 
Management  College’s  20  week  Program  Managers  Course  where  he  has  lectured  for  more  than  12 
years  on  “Aerospace  Materials  in  Acquisition”  and  “Advanced  Manufacturing.” 
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Lean  Six  Sigma  Implementation  for  Military  Logistics  to  Improve 
Readiness 


Presenter:  Keebom  Kang,  Associate  Professor,  joined  the  Naval  Postgraduate  School  in  1988,  where 
he  teaches  supply-chain,  logistics  engineering  and  computer  simulation  modeling  courses  for  the  MBA 
program.  His  research  interests  are  in  the  areas  of  logistics  and  simulation  modeling  in  various  military 
applications.  He  received  his  PhD  in  Industrial  Engineering  from  Purdue  University.  Prior  to  joining  NPS, 
he  was  on  the  faculty  of  the  Industrial  Engineering  Department  at  the  University  of  Miami,  Coral  Gables, 
Florida  (1983-1988).  He  had  held  visiting  professor  positions  at  Syracuse  University  (Summer,  1985), 
Georgia  Institute  of  Technology  (Fall,  2003),  Asia  Institute  of  Technology  in  Thailand  (Winter,  2004),  and 
Pohang  Institute  of  Science  and  Technology  in  Korea  (Spring,  2004). 

Dr.  Kang  has  published  many  theoretical  and  applied  papers  in  Operations  Research,  IEEE  Transactions 
in  Communications,  IIE  Transactions,  Telecommunication  Management,  Naval  Logistics  Research,  and 
other  technical  journals  and  conference  proceedings.  He  was  the  Co-editor  of  the  1995  Winter  Simulation 
Conference  Proceedings  and  the  program  chair  for  the  2000  Winter  Simulation  Conference.  He  served  as 
an  editorial  board  member  for  IIE  Transactions,  an  associate  editor  for  IMA  Journal  of  Mathematics  for 
Management  Science,  an  associate  editor  for  IIE  Transactions  Special  Issue  on  Computer  Simulation, 
and  as  a  referee  for  many  professional  journals,  including  Operations  Research,  Management  Science, 
and  Naval  Logistics  Research.  He  also  served  as  a  proposal-review  panel  member  for  the  National 
Science  Foundation  (1990,  1992,  1994).  He  is  currently  an  Associate  Editor  for  the  Naval  Research 
Logistics  (2006-present). 

Presenter:  Uday  Apte,  is  Professor  of  Operations  Management  at  the  Graduate  School  of  Business  and 
Public  Policy,  Naval  Postgraduate  School,  Monterey,  CA.  Before  joining  NPS,  Dr.  Apte  taught  at  the 
Wharton  School,  University  of  Pennsylvania,  Philadelphia,  and  at  the  Cox  School  of  Business,  Southern 
Methodist  University,  Dallas.  He  is  experienced  in  teaching  a  range  of  operations  management  and 
management  science  courses  in  the  Executive  and  Full-time  MBA  programs. 

Dr.  Apte  has  served  as  a  founder  and  President  of  the  College  of  Service  Operations,  Production  and 
Operations  Management  Society  (POMS),  as  a  board  member  of  POMS,  and  as  guest  editor  of 
Production  and  Operations  Management  journal.  Areas  of  his  research  interests  include  managing 
service  operations,  supply-chain  management,  technology  management,  and  globalization  of  information¬ 
intensive  services.  He  has  published  over  30  articles,  five  of  which  have  won  awards  from  professional 
societies.  His  research  articles  have  been  published  in  prestigious  journals  including  Management 
Science,  Interfaces,  Production  and  Operations  Management,  Journal  of  Operations  Management, 
Decision  Sciences,  IIE  Transactions,  Interfaces,  and  MIS  Quarterly.  He  has  co-authored  two  books, 
Manufacturing  Automation  and,  Managing  in  the  Information  Economy. 

Dr.  Apte  holds  a  PhD  in  Decision  Sciences  from  the  Wharton  School,  University  of  Pennsylvania.  His 
earlier  academic  background  includes  a  MBA  from  the  Asian  Institute  of  Management,  Manila, 

Philippines,  and  Bachelor  of  Technology  (Chemical  Engineering)  from  the  Indian  Institute  of  Technology, 
Bombay,  India. Naval  Postgraduate  School. 

Keebom  Kang,  PhD 
Associate  Professor 

Graduate  School  of  Business  and  Public  Policy 

Naval  Postgraduate  School 

555  Dyer  Road 

Monterey,  CA  93950-5197 

(831)656-3106 

kkanq@nps.edu 
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Abstract 

In  general,  during  the  lifecycle  of  a  weapon  system,  a  significantly  larger  amount  of 
money  gets  spent  on  operating  and  maintaining  the  system  than  on  acquiring  it.  Hence, 
efficient  logistics  systems,  including  transportation,  inventory  management,  modifications  and 
maintenance  activities,  are  critically  important  for  containing  the  lifecycle  costs  of  weapon 
systems  and  for  maintaining  the  highest  level  of  military  readiness  given  the  extant  fiscal 
constraints.  This  paper  describes  Lean  Six  Sigma  (LSS),  a  strategically  important  and  proven 
logistics  initiative  for  both  reduced  lifecycle  costs  and  improved  readiness. 

With  aging  weapon  systems,  the  US  Department  of  Defense  is  facing  ever-increasing 
military  expenses  to  maintain  military  readiness.  Hence,  the  Department  of  Defense  is  keenly 
interested  in  implementing  Lean  Six  Sigma  in  all  the  services.  We  begin  this  paper  by  providing 
an  overview  of  military  logistics  and  discussing  the  critical  concepts  of  readiness  and  cycle-time. 
Thereafter,  we  present  an  overview  of  Lean  Six  Sigma  methodologies — including  Lean 
production  and  Six  Sigma,  and  describe  the  experience  in  implementing  Lean  Six  Sigma  in  the 
Army,  Navy  and  Air  force.  The  paper  ends  with  a  discussion  of  the  managerial  guidelines  for 
successfully  implementing  Lean  Six  Sigma. 

Keywords:  Lean  Six  Sigma,  Lean  Production,  Six  Sigma,  Military  Logistics,  Readiness, 
Lifecycle  Costs 

Introduction 

Three  essential  factors  to  maintaining  strong  military  power  and  readiness  are  well- 
trained  troops/  well-educated  officers,  reliable  high-tech  weapon  systems,  and  well-designed 
logistics  systems  to  support  troops  and  improve  the  readiness  of  the  weapon  systems.  In 
purchasing  weapon  systems,  program  managers  widely  use  acquisition  costs  as  the  primary, 
and  at  times  the  only,  criteria  for  decision-making.  However,  in  general,  during  the  lifecycle  of  a 
weapon  system,  a  significantly  larger  amount  of  money  gets  spent  on  operating  and  maintaining 
the  system  than  on  acquiring  it.  Hence,  efficient  logistics  systems — including  transportation, 
inventory  management,  modifications  and  maintenance  activities — are  critically  important  for 
containing  the  lifecycle  costs  (LCC)  of  weapon  systems  and  for  maintaining  the  highest  level  of 
military  readiness  given  the  extant  fiscal  constraints.  This  paper  will  describe  Lean  Six  Sigma 
(LSS),  a  strategically  important  and  proven  logistics  initiative  for  both  reduced  lifecycle  costs 
and  improved  readiness. 

Two  major  components  of  LCC  are  Acquisition  costs  and  Operations  and  Maintenance 
(O&M)  costs.  Acquisition  costs  include  such  items  as  research,  development,  test  and 
evaluation,  program  management,  engineering  design,  initial  spare  parts,  manufacturing  and 
production,  facilities  and  construction,  and  initial  training.  O&M  costs,  on  the  other  hand, 
include  such  cost  categories  as  labor,  materials,  and  overhead,  operations,  scheduled  and 
unscheduled  maintenance,  training,  replacement  and  renewal,  transportation, 
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system/equipment  modification,  technical  data  collection,  documentation  and  database 
management,  energy  and  facility  usage,  and  disposal  costs.  Without  question,  the  logistics 
systems  have  a  great  deal  of  influence  on  the  size  of  O&M  costs. 

It  is  difficult  to  generalize  the  percentage  of  money  spent  on  operations  and 
maintenance  of  a  typical  weapon  system.  Some  literature  points  out  that  the  O&M  costs 
contribute  to  60%  of  the  total  lifecycle  cost  on  average  (DAU,  2006),  while  other  sources 
estimate  these  costs  to  be  as  high  as  80%  of  the  total  (Cost  Analysis  Improvement  Group, 
1992).  In  any  event,  with  the  Service  Extension  Program  (SEP)  that  many  weapon  systems  are 
experiencing  these  days,  the  percentage  of  the  total  lifecycle  cost  spent  on  O&M  is  simply 
becoming  larger.  Most  weapon  systems  were  originally  designed  for  a  lifecycle  of  20+  years, 
but  some  have  been  stretched  to  last  as  long  as  50  years.  In  the  case  of  B-52  aircraft,  for 
example,  the  lifecycle  is  expected  to  extend  to  80  years,  in  which  case  the  O&M  costs  expect  to 
form  as  much  as  90%  of  its  lifecycle  cost  (Parker,  1999). 

With  aging  weapon  systems,  the  US  Department  of  Defense  (DoD)  is  facing  ever- 
increasing  O&M  costs.  The  DoD  is,  therefore,  keenly  interested  in  applying  Lean  Six  Sigma 
methodologies  to  cut  down  O&M  costs.  Experiences  of  the  private  sector  in  implementing  Lean 
Six  Sigma  illustrate  that  the  methodology  is  as  effective  in  improving  business  processes  as  it  is 
in  improving  the  manufacturing  processes.  Thus,  successful  implementation  of  LSS 
methodologies  would  also  reduce  acquisition  costs  by  improving  acquisition  and  contracting 
processes.  Hence,  in  this  paper  we  will  discuss  Lean  Six  Sigma  and  its  application  in  the 
military.  This  paper  is  organized  as  follows:  in  Section  2,  we  provide  an  overview  of  military 
logistics  and  discuss  the  critical  concepts  of  readiness  and  cycle-time.  In  Section  3,  we 
describe  background  material  for  LSS  methodologies.  Section  4  includes  examples  of  LSS 
implementation  in  the  US  Army,  the  US  Navy  and  the  US  Air  Force.  In  Section  5,  we  conclude 
the  paper  by  presenting  managerial  guidelines  and  by  discussing  the  challenges  present  in 
implementing  LSS  in  the  military. 


Military  Logistics 


Military  Logistics  support  deals  with  everything  required  to  provide  warfighters  with  the 
right  stuff  at  the  right  time  at  the  right  place  at  the  right  cost.  The  goal  of  military  logistics  support 
is  to  maintain  the  highest  possible  level  of  readiness,  commonly  expressed  as  operational 

MTBM  uptime 


availability:  Ao  = 


where  MTBM  is  the  mean  time 


MTMB  +  MDT  uptime  +  downtime 

between  maintenance,  and  MDT  is  the  maintenance  down  time — which  includes  repair  time  and 
administrative  and  logistics  delay  times.  Intuitively,  operational  availability  is  the  fraction  of  time 
a  weapon  system  is  operational  or  mission  capable.  Clearly,  operational  availability  can  be 
improved  by  increasing  MTBM  (i.e.,  increasing  reliability)  and/or  decreasing  MDT  (i.e. ,  reducing 
repair  or  cycle-time).  Thus,  the  two  key  issues  to  improve  weapon  systems  readiness  are 
reliability  improvement  and  cycle-time  reduction. 


From  Little's  Law  (Little,  1961),  reducing  repair  or  cycle-time  reduces  pipeline  inventory 
directly,  and  leads  to  significant  savings  in  inventory  costs.  The  relationship  between  repair  or 
cycle-time  and  inventory  levels  is  critically  important  (yet,  troublesome)  in  the  military  because  it 
crosses  physical,  organizational,  and  financial  barriers.  Inventory  managers  strive  to 
consolidate  and  minimize  stocks  of  piece-parts  to  free-up  resources  for  other  priorities.  They 
also  seek  to  get  quick  turnaround  on  repairable  components  in  order  to  minimize  pipeline 
inventory.  However,  stockout  of  spare  parts  or  consumable  components  results  in  delays  in 
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repair  processes  and,  eventually,  serious  readiness  degradation.  Cycle-time  reduction  in  a 
military  logistics  channel  (repair  depots,  intermediate-level  maintenance,  inventory  control 
points,  and  supply  centers)  also  means  that  more  weapon  systems  are  available  in  the  field  or 
fleet.  On  the  other  hand,  increased  cycle-time  causes  a  vicious  cycle  of  deteriorating  military 
readiness.  For  instance,  poor  logistics  support  (e.g.,  lack  of  spare  parts,  personnel,  and/or 
training)  increases  the  cycle-time,  which  in  turn  decreases  readiness,  Ao.  Therefore,  the 
warfighters  are  forced  to  satisfy  mission  requirements  with  a  fewer  number  of  mission-capable 
weapon  systems,  resulting  in  stress  on  those  fewer  mission-capable  systems.  Due  to  this 
stress,  more  system  failures  occur,  which  in  turn  generate  more  workload  at  repair  facilities. 
Thus,  the  repair  turnaround  time  can  become  even  longer.  And  the  vicious  cycle  can  go  on. 

The  following  simple  example  explains  the  importance  of  cycle-time  reduction  in  military 
logistics.  Suppose  that  the  US  Navy  has  800  F/A-18  Hornet  aircraft,  each  of  which  costs  $50 
million,  and  that  the  Standard  Depot  Level  Maintenance  (SDLM)  is  done  every  4  years.  If  the 
MDT  is  one  year,  the  readiness,  Ao,  will  be  4/(4+1)  =  0.8.  Thus,  only  80%  of  800,  or  640, 
aircraft  will  be  mission-capable  on  average  since  an  aircraft  would  be  available  for  mission  for 
four  years  (and  at  the  depot  for  one  year)  out  of  every  five  years.  This  also  means  160  aircraft 
will  be  non-mission  capable  at  any  given  time.  If  the  MDT  can  be  reduced  to  6  months,  Ao  will 
be  0.889;  or,  only  89  instead  of  160  aircraft  will  be  at  the  depot  for  maintenance  at  any  given 
time.  It  is  equivalent  of  having  71  additional  aircraft  (worth  more  than  $3.5  billion)  in  the  fleet. 

On  the  other  hand,  if  having  640  mission-capable  aircraft  available  is  adequate,  it  would  mean 
reducing  the  fleet  size  by  80  aircraft  and  freeing  up  $4  billion  expenditure  for  other  purposes. 
See  Kang,  Gue  and  Eaton  (1998)  for  a  cycle-time  reduction  case  study  at  a  Navy  depot. 

The  Department  of  Defense  and  its  services  have  many  on-going  initiatives  to  cut  down 
maintenance  cycle-time  to  improve  military  readiness.  The  Navy  has  been  working  on  the  Sea 
Based  Logistics  to  cut  down  distribution  time  by  supporting  “customers”  on  shore  directly  from 
the  sea  by  eliminating  “Iron  Mountains”  (middlemen)  in  the  supply-chain  management  context. 
Likewise,  since  1995,  the  US  Army  has  implemented  Velocity  Management  (Dumond  et  al., 
2001)  which  focuses  on  improving  the  speed  and  accuracy  with  which  materials  and  information 
flow  from  factories  to  fox  holes.  The  US  Air  Force  has  implemented  Agile  Logistics,  and  the  US 
Marine  Corps,  Precision  Logistics  for  cycle-time  reduction. 

More  recently,  all  branches  of  the  US  military,  Army,  Navy  and  the  Air  Force,  are  actively 
applying  Lean  Six  Sigma  methodology  to  their  various  activities  to  reduce  cycle-time  and  to 
reduce  maintenance  expenses.  We  will  describe  the  details  of  current  initiatives  of  Lean  Six 
Sigma  in  the  military  services  in  Section  4. 

Levels  of  Maintenance 

We  can  use  the  US  Navy’s  aviation  maintenance  system  to  understand  how  military 
maintenance  logistics  are  typically  conducted.  The  Naval  Aviation  Maintenance  Program 
divides  maintenance  into  three  levels:  organizational  level  (O-level),  intermediate  level  (l-level), 
and  depot  level  (D-level),  which  are  similar  in  structure  to  multi-echelon  logistics  support 
systems  of  commercial  firms  (e.g.,  Blanchard,  2004)  or  other  services.  To  achieve  economies 
of  scale  in  maintenance  equipment  and  personnel,  levels  of  maintenance  are  made 
progressively  more  capable,  with  D-level  being  the  most  capable.  However,  the  longer 
turnaround  time  at  D-level  also  increases  the  work-in-process  and  requires  more  spare  parts  to 
maintain  the  desired  readiness  level. 

O-level  maintenance  is  performed  at  the  site  and  typically  involves  simple  repairs  or  the 
replacement  of  modular  components,  l-level  maintenance  involves  more  difficult  repairs  and 
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maintenance,  including  the  repair  and  testing  of  modules  that  have  failed  at  the  O-level.  l-level 
maintenance  for  Navy  aircraft  is  done  at  Aircraft  Intermediate  Maintenance  Departments 
(AlMDs)  ashore  in  naval  air  stations  or  afloat  in  aircraft  carriers.  D-level  maintenance  activities, 
called  Naval  Aviation  Depots  (NADEPs),  ensure  the  continued  flight  integrity  and  safety  of 
airframes  and  related  flight  systems  throughout  their  service  lives.  This  involves  performing 
maintenance  beyond  the  capabilities  of  the  lower  levels,  usually  on  equipment  requiring  major 
overhaul  or  rebuilding  of  end-items,  subassemblies,  and  parts.  The  Navy  operates  three 
NADEPs  in  the  US  (North  Island,  CA;  Cherry  Point,  NC;  and  Jacksonville,  FL)  and  fleet  repair 
facility  sites  in  Italy  and  Japan. 

The  repair  cycle  begins  when  an  unserviceable  repairable  is  turned  for  maintenance, 
and  it  ends  when  the  item  is  recorded  on  the  inventory  control  point  records  as  being  ready-for- 
issue  (RFI).  Repair  cycle-time  includes  shipping  and  processing  time,  accumulation  time,  repair 
time,  time  awaiting  parts,  and  delivery  time.  Unserviceable  items  may  remain  in  storage  for 
extended  times  for  various  reasons. 

Readiness  and  Inventory  Management 

Aviation  readiness  is  measured  by  computing  fully  mission-capable  (FMC)  rates.  The 
FMC  rate  indicates  the  operational  availability  of  the  aircraft  in  a  unit — that  is,  the  fraction  of 
aircraft  that  are  mission  capable  at  any  arbitrary  time.  When  aircraft  are  partially  mission 
capable  or  not  mission  capable,  it  is  because  of  either  maintenance  or  supply  problems. 

Aviation  items,  especially  repairables,  are  very  expensive  to  maintain.  For  example, 
each  aircraft  carrier  carries  onboard  an  Aviation  Consolidated  Allowance  List  (AVCAL) 
consisting  of  consumable  and  repairable  items  and  subassemblies  required  to  support  the  Air 
Wing  for  90  days  of  wartime  operations.  A  typical  AVCAL  consists  of  tens  of  millions  of  line 
items  valued  at  hundreds  of  millions  of  dollars.  Repairable  items  represent  only  1 0%  of  the  total 
line  items,  but  90%  of  the  total  value  of  the  AVCAL  (USS  Independence  Shipboard  Uniform 
Automatic  Data  Processing  System  Report  008,  1991,  July  26). 

Material  readiness  demands  spare  parts,  but  fiscal  constraints  have  put  pressure  on  the 
Navy  to  reduce  inventory  levels  at  AlMDs  and  stock  points.  The  two-part  solution  is  easier  said 
than  done:  select  a  “better”  mix  of  spares  and  reduce  repair  cycle-time.  Both  tend  to  improve 
readiness  for  a  given  cost  or  achieve  the  same  readiness  for  lower  cost. 

The  relationship  between  spares/inventory  levels  and  cycle-time  is  a  key  to 
understanding  how  to  achieve  higher  readiness  at  lower  cost.  Kang  (1993)  shows  the 
diminishing  marginal  utility  of  spare  parts,  implying  that  additional  spare  parts  beyond  a  certain 
threshold  level  will  not  improve  readiness.  Those  additional  spare  parts,  once  they  are  turned  in 
after  failure,  will  simply  increase  the  work-in-process  or  inventory  at  repair  facilities.  Spares 
levels  and  repair  cycle-time  must  be  considered  together  when  attempting  to  improve  material 
readiness  (see  Kang  &  Gue,  1997). 

During  the  past  30  years,  the  military  has  been  implementing  spares  methodologies 
based  on  the  readiness-based  METRIC  models  such  as  those  described  in  Sherbrooke  (1992). 
Rather  than  the  traditional  approach  to  inventory  problems  that  minimize  holding  and  ordering 
costs  for  individual  items  subject  to  a  service  level,  readiness-based  models  seek  to  maximize 
Ao  for  multiple  items  directly  and  simultaneously,  subject  to  a  budget  constraint.  It  is  possible  to 
measure  Ao  for  a  specific  component,  such  as  an  aircraft  engine,  as  opposed  to  measuring  Ao 
for  the  aircraft  itself.  An  improvement  in  Ao  for  the  engine  will  provide  some  marginal 
improvement  in  Ao  for  the  aircraft.  But  this  improvement  will  not  be  one-to-one:  large 
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improvements  in  engine  availability  may  yield  only  trivial  improvements  in  aircraft  availability, 
depending  not  only  on  the  failure  rate  of  the  engines,  but  on  the  performance  and  availability  of 
all  the  other  critical  components  of  the  aircraft.  The  readiness-based  models  are  important  to 
military  systems  because  they  treat  all  of  the  critical  components  in  a  weapon  system  together 
in  order  to  achieve  the  singular  objective  of  maximizing  the  Ao  of  the  weapon  system. 
Implementation  of  these  models  requires  detailed,  accurate  information  about  the  reliability  of 
components,  but  the  rewards  have  been  worth  the  effort  in  many  systems.  For  example, 
Sherbrooke  (1992)  reports  inventory  investment  being  cut  nearly  in  half,  with  no  degradation  in 
readiness,  during  a  test  for  the  Air  Force.  Hale  (1994)  also  shows  significant  inventory  savings 
in  the  Navy  after  implementing  readiness-based  models. 

Lean  Six  Sigma 

Penchant  for  process  improvement  is  inherent  in  human  nature;  even  our  distant 
ancestors  discovered  a  better  way  to  start  fire,  make  arrowheads  and  spears,  or  build  shelters 
(Dershin,  2004).  Early  improvements  probably  came  about  through  trial  and  error  and  took 
hundreds  (if  not  thousands)  of  years  to  become  part  of  the  human  skill  set.  Almost  up  to  the 
modern  times,  such  improvements  were  the  carefully  guarded  secrets  of  the  select  few. 
However,  the  fast  pace  of  modern  commercial/industrial  economy  has  given  rise  to  the 
structured  problem-solving  methodologies  for  process  improvement  that  are  well  understood  by 
and  available  to  all. 

Two  major  approaches  for  structured  problem  solving  emerged  separately  in  the  20th 
century  and  have  come  to  be  known  as  “Lean”  and  “Six  Sigma”  methodologies.  Lean 
improvements  focus  on  process  speed  and  waste  removal,  while  Six  Sigma,  like  its  predecessor 
Total  Quality  Management  (TQM),  focuses  on  the  removal  of  process  defects  and  the  reduction 
of  process  variability.  Ironically,  Six  Sigma  and  Lean  have  often  been  regarded  as  rival 
initiatives.  Lean  enthusiasts  note  that  Six  Sigma  pays  little  attention  to  anything  related  to  speed 
and  flow,  while  Six  Sigma  supporters  point  out  that  Lean  fails  to  address  key  concepts  like 
customer  needs  and  process  variation.  To  some  extent,  these  are  valid  arguments.  Yet,  they 
have  been  more  often  used  by  the  practitioners  to  promote  the  choice  of  one  versus  the  other 
approach.  However,  today’s  need  for  an  even  higher  level  of  competitiveness  than  that 
achieved  through  implementing  either  methodology  has  now  convinced  practitioners  that  these 
two  approaches  are  synergistic,  and  there  is  benefit  to  be  realized  by  blending  the  two. 
Therefore,  in  the  new  millennium,  we  are  witnessing  the  emergence  of  Lean  Six  Sigma 
(George,  2002;  Nash,  Poling  &  Ward,  2006). 

Lean  and  Six  Sigma  are  two  different  bodies  of  knowledge.  The  Six  Sigma  is  all  about 
locating  and  eliminating  root  causes  of  process  problems.  The  Six  Sigma  tools,  such  as  the  “the 
five  whys,”  are  designed  to  find  the  root  cause/s  of  the  problems  and  build  models  of  cause  and 
effect.  The  process  is  then  redesigned  with  the  root  cause/s  eliminated. 

Lean  is  different.  As  popularized  by  Womack  and  Jones  (2003),  the  Lean  roadmap  is  one  of 
successive  refinements  to  improve  the  overall  process  through  the  following  steps  (Apte  &  Goh, 
2004): 


■  Specify  value  in  the  eyes  of  the  customer 

■  Identify  the  value  stream  and  eliminate  waste 

■  Make  value  flow  at  the  pull  of  the  customer 
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■  Involve  and  empower  employees 

■  Continuously  improve  in  the  pursuit  of  perfection. 

Since  Lean  Six  Sigma  is  a  synergistic  blending  of  Lean  Production  and  Six  Sigma 
methodologies,  we  will  present  a  brief  overview  of  these  two  methodologies. 

Lean  Production 

Lean  can  be  defined  as  a  set  of  principles  and  tools  that  helps  us  eliminate  process 
activities  that  don't  add  value,  and  create  "flow"  in  a  process  (Dennis,  2002).  A  Lean  process  is 
defined  as  one  that  uses  only  the  absolute  minimum  of  resources  to  add  value  to  the  service  or 
product.  Lean  manufacturing  can  also  be  viewed  as  a  management  philosophy  focusing  on 
reduction  of  the  eight  types  of  wastes  (Human  Talent,  Over-production,  Waiting  time, 
Transportation,  Processing,  Inventory,  Motion  and  Scrap)  in  manufacturing  or  service  processes 
(“Lean  Manufacturing,”  2006).  By  eliminating  waste  (muda),  quality  is  improved,  production  time 
is  reduced,  and  cost  is  reduced.  Lean  "tools"  include  continuous  process  improvement  (kaizen), 
"pull"  production  process  (by  means  of  kanban)  and  mistake-proofing  (poka-yoke).  Lean,  as  a 
management  philosophy,  is  also  very  focused  on  creating  a  better  workplace  through  the 
Toyota  principle  of  "respect  for  humanity." 

Origins  of  Lean  Production  can  be  traced  to  the  Scientific  Management  principles  of 
Frederic  Taylor  (1911)  and  to  the  practical  genius  of  Henry  Ford  (Levinson,  2002).  But  the 
principles  of  Lean  Production  were  more  fully  embodied  in  its  recent  incarnations:  Just  in  Time 
Systems  and  Toyota  Production  System  (Ohno,  1988).  The  term  Lean  Production  was  coined 
by  Womack,  Jones  and  Roos  (1991)  in  their  best  seller,  The  Machine  that  Changed  the  World. 
The  book  chronicles  the  transitions  of  automobile  manufacturing  from  craft  production  to  mass 
production  to  lean  production.  “Theory  of  Constraints  (TOC)”  popularized  by  Goldratt  and  Cox 
(1992)  in  their  novel  The  Goal  is  also  typically  used  in  implementing  Lean  production.  Simply 
put,  TOC  involves  identification  and  use  of  the  bottleneck  (i.e. ,  the  constraint)  of  the  system  to 
set  the  operational  pace  of  the  system’s  components  and  to  achieve  a  synchronous  flow  so  as 
to  maximize  the  throughput  (i.e.,  the  money-making  potential)  of  the  system. 

At  the  heart  of  Lean  is  the  determination  of  value.  Value  is  defined  as  form,  feature  or 
function  for  which  a  customer  is  willing  to  pay.  The  processes  that  do  not  add  value  are  deemed 
waste.  The  Lean  framework  is  used  as  a  tool  to  focus  resources  and  energies  on  producing  the 
value-added  features  while  identifying  and  eliminating  non-value  added  activities.  Processes  in 
Lean  are  thought  of  as  value  streams.  Lead-time  reduction  and  the  flow  of  the  value  streams 
are  the  major  areas  of  focus  in  Lean.  Value-stream  mapping  helps  teams  understand  the  flow  of 
material  and  information  in  creating  and  delivering  the  product  or  services  being  offered  to  the 
customer  by  the  organization. 

In  summary,  in  its  current  implementation,  the  Lean  methodology: 

■  Provides  tools  for  analyzing  process  flow  and  delay  times  at  each  activity  in  a  process, 

■  Emphasizes  Value-stream  Mapping,  which  centers  on  the  separation  of  "value-added" 
from  "non-value-added"  work  with  tools  to  eliminate  the  root  causes  of  non-valued 
activities  and  their  cost, 

■  Uses  Theory  of  Constraints  as  its  integral  element  to  identify  bottlenecks  and  achieve  a 
synchronous  flow  in  the  system, 
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■  Recognizes  and  attempts  to  eliminate  8  types  of  waste/non-value-added  work:  defects, 
inventory,  over-production,  waiting  time,  motion,  transportation,  processing,  and  human 
talent,  and 

■  Creates  workplace  organization  through  Five  S  methodology  consisting  of  sort, 
straighten,  sustain,  sweep,  and  standardize. 

Six  Sigma 

Six  Sigma  is  a  management  technique  that  aims  to  develop  and  deliver  near-perfect 
products  and  services.  The  primary  goal  of  Six  Sigma  is  to  improve  customer  satisfaction  (and, 
thereby,  profitability)  by  reducing  and  eliminating  defects.  In  this  case,  the  defects  may  be 
related  to  any  aspect  of  customer  satisfaction:  product  quality,  delivery  performance,  and 
product  cost.  Six  Sigma  is  targeted  at  reducing  variation  in  a  business  processes.  It  can  also  be 
a  great  way  to  permeate  the  culture  of  continuous  improvement  in  an  organization. 

The  term  "Six  Sigma"  refers  to  a  statistical  construct  that  measures  how  far  a  given 
process  deviates  from  perfection.  A  level  of  Six  Sigma  (about  3.4  defects  per  every  million 
items)  represents  the  highest  level  of  quality:  virtually  all  products  and  business  processes  are 
defect-free.  It  should  be  noted  that  most  companies  today  function  at  only  a  three  or  four  sigma 
level  and  lose  10-15%  of  their  total  revenue  due  to  defects.  Thus,  a  typical  company  stands  to 
benefit  significantly  from  implementing  Six  Sigma. 

Six  Sigma  originated  in  1986  with  the  efforts  of  Bill  Smith,  a  senior  engineer  and  scientist 
at  Motorola  (McCarty,  2004).  It  was  originally  used  to  improve  manufacturing  processes  at 
Motorola.  While  Six  Sigma  has  its  roots  in  the  total  quality  management  (TQM)  approach  of  the 
1980s,  today  it  is  much  more  than  that.  It  is  now  being  used  across  a  wide  range  of  industries, 
including  banking,  insurance,  telecommunications,  construction,  healthcare,  and  software. 
Interestingly,  the  methodology  gained  industry-wide  acceptance  in  the  mid-90s  when  Jack 
Welch,  CEO  of  GE,  successfully  launched  it  within  the  entire  company  (General  Electric,  2006) 
and  began  vouching  for  the  billion-dollar  benefits  realized  by  GE  through  the  use  of  Six  Sigma 
methodology.  For  instance,  in  1999  alone,  GE  reported  that  it  saved  $2  billion  using  Six  Sigma 
principles. 

In  Six  Sigma  applications  in  service-sector  industries,  the  program  implies  going  beyond 
the  highest  quality  level  targeted  in  the  manufacturing  process.  For  example,  an  average  of  3.4 
errors  in  every  one  million  financial  transactions  would  not  be  acceptable  to  a  financial 
institution.  Six  Sigma  now  has  much  broader  meaning.  Simply  put,  Six  Sigma: 

■  Emphasizes  the  need  to  recognize  opportunities  and  eliminate  defects  as  defined  by 
customers, 

■  Recognizes  that  process  variation  hinders  our  ability  to  reliably  deliver  high-quality 
services, 

■  Requires  data-driven  decisions  and  incorporates  a  comprehensive  set  of  quality  tools 
under  a  powerful  framework  for  effective  problem  solving,  and 

■  Provides  a  highly  prescriptive  cultural  infrastructure  effective  in  obtaining  sustainable 
results. 

In  any  improvement  project,  utilization  of  a  well-defined  improvement  procedure  is  critically 
important.  The  most  commonly  used  standard  improvement  procedure  in  Six  Sigma  is  DMAIC 
(Define,  Measure,  Analyze,  Improve  and  Control).  DMAIC  is  a  structured,  disciplined,  rigorous 
approach  to  process  improvement  consisting  of  the  five  phases,  in  which  each  phase  is  linked 
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logically  to  the  previous  phase  as  well  as  to  the  next  phase.  A  detailed  description  of  these 
phases  can  be  found  in  Stamatis  (2004)  and  Rath  and  Strong  (2006). 

In  terms  of  the  tools  and  techniques  used  for  process  improvement,  there  is  only  a  marginal 
difference  between  Six  Sigma  and  the  Total  Quality  Management  approaches.  But  what  sets 
Six  Sigma  apart  from  TQM,  which  is  perhaps  the  most  important  reason  behind  the  success  of 
Six  Sigma,  is  the  establishment  of  organizational  infrastructure  for  ensuring  continuous  process 
improvement.  Thus,  Six  Sigma  should  be  ideally  viewed  as  a  management  system  that 
integrates  strategic  objective  and  measurement  systems  development,  and  provides  the 
guidance  for  project  prioritization  and  governance.  It  is  a  performance-management  system  to 
drive  a  more  focused  execution  of  the  overall  business  strategy.  The  essential  premise  of  the 
Six  Sigma  Management  System  is  that  there  is  a  leadership  team  in  place  whose  members  are 
willing  and  capable  of  engaging  in  a  disciplined,  team-based  process  of  continuously  monitoring 
real-time  organizational  performance  metrics  and  then  taking  action  in  the  form  of  project 
reviews.  The  team  engages  in  frequent  dialogue  regarding  performance  related  to  customer  and 
market  requirements  as  well  as  performance  related  to  critical  improvement  projects.  As  a  result 
of  these  efforts,  an  organization-wide  dialogue  is  created  that  drives  top-to-bottom  focus  on 
daily  execution  and  a  culture  of  continuous  improvement. 

Six  Sigma  identifies  five  key  organizational  roles  for  its  successful  implementation  (“Six 
Sigma,”  2006): 

■  Executive  Leadership  includes  CEO  and  other  key  top  management  team  members. 
They  are  responsible  for  establishing  a  vision  for  Six  Sigma  implementation. 

■  Champions  are  responsible  for  the  Six  Sigma  implementation  across  the  organization  in 
an  integrated  manner. 

■  Master  Black  Belts,  identified  by  Champions,  act  as  in-house,  full-time,  expert  coaches 
for  the  organization  on  Six  Sigma  initiatives. 

■  Black  Belts  operate  under  Master  Black  Belts  to  apply  Six  Sigma  methodology  to 
specific  projects.  They  devote  100%  of  their  time  to  Six  Sigma.  They  primarily  focus  on 
Six  Sigma  project  execution,  whereas  Champions  and  Master  Black  Belts  focus  on 
identifying  projects/functions  for  Six  Sigma. 

■  Green  Belts  are  the  employees  who  take  up  Six  Sigma  implementation  along  with  their 
other  job  responsibilities.  They  operate  under  the  guidance  of  Black  Belts  and  support 
them  in  achieving  the  overall  results. 

Please  note  that  there  exists  a  large  variation  in  the  way  the  above  roles  are  defined  and 
utilized  within  the  Six  Sigma  implementations  in  different  enterprises  and  that  specific  training 
programs  are  available  to  train  people  to  fulfill  these  roles. 

Lean  Six  Sigma 

As  noted  earlier,  the  process  improvement  methods  of  Lean  and  Six  Sigma  have  been 
practiced  separately  for  many  years.  However,  in  recent  years,  practitioners  have  come  to 
realize  that  the  two  methodologies  are,  in  fact,  dependent  on  each  other  for  greater  success. 

For  example,  it  is  impossible  to  run  a  process  with  minimum  waste  or  at  a  dependable  capacity 
if  individual  process  steps  are  highly  variable.  On  the  other  hand,  one  can  carefully  study  the 
complex  processes,  looking  for  root  causes  using  elegant  statistical  techniques,  and  never 
make  improvements  in  cycle-time  or  productivity  that  can  be  obtained  from  value-stream 
analysis. 
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To  the  extent  Lean  and  Six  Sigma  approaches  have  their  own  strengths  and 
weaknesses,  the  specific  action  plan  to  be  followed  in  effectively  implementing  Lean  Six  Sigma 
(for  example,  Lean  first  followed  by  Six  Sigma  later  or  vice  versa)  is  dependent  on  the  nature  of 
the  situation  at  hand.  For  example,  the  problems  related  to  accuracy  and/or  completeness  are 
usually  addressed  best  by  the  tools  of  Six  Sigma;  consequently,  those  tools  should  be 
introduced  first.  However,  if  the  customer  needs  quick  results,  and  if  the  problem  is  related  to 
timeliness  or  productivity,  Lean  should  be  implemented  first  with  an  understanding  that  deep 
and  complex  problems  will  be  solved  only  by  the  subsequent  use  of  the  Six  Sigma  tools. 

In  summary,  Lean  and  Six  Sigma  are  rich  bodies  of  knowledge  and  are  mature 
methodologies  for  solving  a  broad  variety  of  process-related  problems.  Each  methodology  has 
its  own  approach  to  process  improvement  and  its  own  tool  set.  Although  Lean  and  Six  Sigma 
methodologies  can  be  mastered  independently,  they  can  and  should  be  implemented  together 
to  realize  the  full  benefits  of  process  improvements  by  any  organization. 

Examples  of  Lean  Six  Sigma  Implementations  in  the  Military 

The  combination  of  Lean  Thinking  and  Six  Sigma  has  proven  to  be  a  very  effective  tool 
in  the  private  sector.  The  success  realized  by  top  companies  such  as  Toyota  and  GE  has 
inspired  the  use  of  Lean  Six  Sigma  in  the  US  Department  of  Defense  (DoD).  Although  the  DoD 
has  implemented  a  number  of  process-improvements  methodologies  with  varying  degrees  of 
success  in  the  past  decade,  it  has  begun  to  explore  the  potential  of  implementing  Lean  Six 
Sigma  throughout  the  entire  DoD  only  recently.  The  early  results  are  very  promising.  As  the 
lean  Six  Sigma  mindset  continues  to  grow  among  the  DoD  community  and  both  the  Lean  and 
Six  Sigma  practices  become  more  commonplace,  the  equipment  and  personnel  available  to  the 
DoD  will  provide  considerably  more  capability  per  taxpayer  dollar  than  ever  before.  We  discuss 
below  some  examples  of  Lean  Six  Sigma  implementations  in  the  US  Army,  Navy  and  the  Air 
Force. 

Army  Implementations 

Faced  with  the  expectations  of  a  shrinking  defense  budget,  the  Secretary  of  the  Army 
Francis  Harvey  signed  an  order  in  March  of  2005  that  would  implement  Lean  Six  Sigma  across 
the  entire  service.  Currently,  several  organizations  within  the  Army  are  implementing  Lean  Six 
Sigma  and  are  enjoying  remarkable  results. 

The  Red  River  Army  Depot  Repair  Facility  is  one  such  organization  (Donnelly,  2006).  In 
implementing  Lean  Six  Sigma,  the  Red  River  Depot  has  made  many  changes  to  its  HMMWV 
repair  line,  such  as:  forming  an  assembly-line  process,  using  time-managed  intervals  to  control 
the  flow  of  work,  organizing  employees  based  on  experience  and  proficiency,  cleaning  up  and 
improving  the  overall  work  environment,  stocking  more  and  better  quality  parts  to  reduce  stock¬ 
outs,  and  training  employees  to  ensure  there  is  no  break  in  continuity  on  the  assembly  line. 
Improvement  efforts  have  resulted  in  the  ability  to  turn  out  32  mission-ready  HMMWV’s  a  day, 
compared  with  three  a  week  in  2004.  The  Lean  process  has  also  lowered  the  cost  of  repair  for 
one  vehicle  from  $89,000  to  $48,000.  Some  of  the  biggest  improvement  ideas  have  come  from 
the  front-line  employees  themselves. 

Other  Army  facilities  boast  similar  progress  as  the  result  of  Lean  Six  Sigma 
methodologies.  Pine  Bluff  Arsenal  in  Arkansas  reduced  its  repair  cycle-time  by  about  90%  and 
increased  its  production  rate  by  about  50%  on  M-40  protective  gas  masks.  Letterkenny  Army 
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Depot  in  Pennsylvania  has  saved  $1 1 .9  million  in  the  cost  of  building  the  Patriot  air-defense 
missile  system.  In  the  Corpus  Christi  Army  Depot,  the  overhaul  time  for  one  T700  helicopter 
engine  was  reduced  by  64%.  These  depots  improved  the  consistency  of  their  repair  operations 
by  increasing  the  mean  time  between  the  engine  overhauls  from  309  hours  to  over  900  hours 
and  improved  the  return  to  field  accuracy  to  above  90%  (Moorman,  2005). 

Despite  these  early  successes,  the  long-term,  future  and  the  resulting  benefits  of  Lean 
Six  Sigma  are  far  from  certain.  Ultimately,  the  key  ingredient  for  the  successful  implementation 
of  Lean  Six  Sigma  is  not  simply  an  order  from  the  top,  but  the  ability  of  commanders  to  change 
the  organization’s  culture  and  convince  the  soldiers  and  employees  that  Lean  Six  Sigma  does 
work  and  that  it  is  worth  the  effort.  The  Red  River  Depot  has  taken  a  small,  yet  interesting,  step 
to  change  the  culture  of  the  organization  by  posting  a  black  cutout  figure  of  a  soldier  with  a 
helmet  and  rifle  with  a  sign  affixed  to  it  that  reads,  “We  build  it  as  if  our  life  depends  on  it.  Theirs 
do!”  This  is  to  serve  as  a  reminder  that  their  job  is  about  more  than  a  paycheck,  and  the  better 
they  can  do  their  job,  the  more  lives  they  can  save. 

Navy  Implementations 

The  AIRSpeed  program  is  perhaps  the  best  known  implementation  of  Lean  Six  Sigma  in 
the  US  Navy.  As  stated  by  the  Secretary  of  the  Navy  Donald  Winter  in  a  memorandum  in  May 
2006,  “Lean  Six  Sigma  (LSS)  is  a  proven  business  process  that  several  elements  of  the  Navy 
and  Marine  Corps  have  initiated  including  training  over  500  Black  Belts  and  1500  Green  Belts 
who  have  facilitated  2800  events  and  projects.  These  activities  have  averaged  a  4:1  return  on 
investment.”  The  following  examples  demonstrate  some  success  stories  in  the  implementation 
of  AIRSpeed. 

a.  In  October  2005,  Naval  Air  Warfare  Center  (NAWC)  accounting  practices  yielded  an 
annual  savings  of  $176.9K,  with  an  additional  anticipated  saving  of  $146. 3K  in  waste 
elimination. 

b.  Since  April  2004,  Aviation  Intermediate  Maintenance  Division  (AIMD)  Whidbey  Island 
reduced  J-52  aircraft  engine  repair  time  from  468  hours  to  233  hours  and  reported 
significant  inventory  and  operating  cost  savings.  Since  February  2006,  AIMD  Patuxent 
River  has  seen  increased  savings  due  to  a  10%  inventory  reduction  and  a  reallocation  of 
166  hours  of  full-time  employees. 

c.  In  June  2006,  Naval  Aviation  Systems  Command’s  (NAVAIR)  PMA  offices  began 
replicating  successes  of  other  PMA  offices,  including  one  office  that  saw  an  estimated 
$163K/year  savings  due  to  reducing  processing  time  from  the  240-days  average  to  a 
predicted  average  of  15  days. 

The  successes  are  due,  in  large  part,  to  the  training  received  by  the  employees  that 
emphasizes  the  use  of  DMAIC  (Define,  Measure,  Analysis,  Improve  and  Control)  methodology 
for  process  improvement.  AIRSpeed  attempts  to  create  an  enterprise-wide,  continuous  process- 
improvement  environment  through  the  incorporation  of  commercial  business  practices.  The 
goal  of  AIRSpeed  is  to  operationalize  cost-wise  readiness  across  the  Naval  Aviation  Enterprise. 

There  are  five  anticipated  long-term  benefits  of  AIRSpeed : 

1 .  Reduce  total  cost  of  Naval  Aviation  by  reducing  inventory,  manpower  and  operating 
expenses. 

2.  Support  the  Fleet  Response  Plan  by  providing  aircraft  Ready  for  Tasking  (RFT). 
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3.  Integrate  the  Maintenance  and  Supply  Support  System  to  provide  seamless  support  to 
the  Fleet. 

4.  Improve  logistics  and  maintenance  response  by  reducing  cycle-time  and  the  logistics 
footprint. 

5.  Place  ownership  and  accountability  at  the  appropriate  levels. 

Air  Force  Implementations 

Over  the  next  several  years,  the  Air  Force  (AF)  is  expected  lose  approximately  40,000 
personnel.  This  loss  of  manpower  means  airmen  must  work  smarter  and  leaner.  Senior  AF 
leadership  has  decided  to  utilize  the  Lean  Six  Sigma  strategy  to  accomplish  this.  Accordingly, 
the  USAF  has  created  a  new  program  office,  Air  Force  Smart  Operations  21  (AFS021),  at  the 
Pentagon  with  Brig.  Gen.  S.  Taco  Gilbert  as  the  Director  of  the  AFS021  Office  (Lopez,  2006). 

The  AF  already  has  several  examples  of  AFS021  at  work.  AF  Materiel  Command  has 
applied  AFS021  and  returned  100  aircraft  to  duty,  as  well  as  reduced  C-5  maintenance  time  by 
50%.  USAF  Europe  (USAFE)  applied  AFS021  practices — they  reduced  the  number  of 
telephone  operators  by  approximately  16%  and  saved  the  command  $2.4  million  (Lopez,  2006). 
The  AF  has  also  begun  implementation  of  Lean  Six  Sigma  concepts  to  their  contracting 
activities.  The  goal  is  to  reduce  the  cycle-time  required  to  award  a  contract  in  support  of  new 
operational  requirements.  The  Global  Hawk  team  followed  the  Lean  Thinking  concepts  to  break 
down  the  contracting  process  into  a  value  stream.  They  identified  steps  that  do  not  add  value 
and  eliminated  them.  By  eliminating  those  unnecessary  steps,  the  process  times  in  three  steps 
of  the  contracting  process  were  cut  by  37%,  40%  and  73%! 

Managerial  Implications 

The  experiences  in  implementing  Lean  Six  Sigma  in  the  military  have  uncovered  several 
valuable  lessons  and  managerial  guidelines.  They  are  briefly  presented  below. 

Active  support  of  senior  leaders  is  a  necessity. 

■  Articulate  clearly  the  need  for  change. 

■  Commit  to  the  change — make  it  last  through  leadership  turnover. 

■  Change  and  accountability  should  be  driven  from  the  top. 

■  Actions  speak  louder  than  words — participate  in  the  effort. 

Initial  successes  are  critically  important. 

■  Carefully  choose  initial  projects. 

■  Assign  high-potential  employees  to  those  projects. 

■  Provide  financial  and  personnel  resources  to  ensure  success. 

■  Initial  successes  turn  the  skeptics  into  believers. 

Emphasize  continuing  education  and  training. 

■  Deploy  1%  of  workforce  as  full-time  Black  Belt  plus  Green  Belts,  Champions,  etc. 
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■  Black  Belts  should  be  selected  from  “future  leaders  of  the  organization.” 

■  Create  Master  Black  Belts  to  take  over  training  at  all  levels. 

■  Senior  organization  leaders  must  be  trained  and  engaged  in  project  selection. 

■  Include  Productivity  Improvement  Training  in  Leadership  Development  Programs. 

Monitor  the  Lean  Six  Sigma  projects. 

■  Assign  concrete  goals  to  project  leaders  and  hold  them  accountable  for  project  results. 

■  Provide  stable  funding  to  ensure  long-term  success. 

■  Demand  validated  return  on  investment;  Keep  score  in  public. 

■  Promote  a  philosophy  that  it  is  OK  to  save  a  dollar  and  give  it  up — it’s  not  your  money. 

■  Middle  management  is  likely  to  provide  the  most  resistance — actively  manage  their 

participation  (increase  the  ratio  between  those  that  get  it  and  those  that  don’t). 

The  LSS  methodology  was  developed  in  the  private  sector.  To  the  extent  the 
competitive  environment,  the  organizational  culture,  and  the  nature  of  operational  challenges 
are  different  in  private-sector  firms  than  in  the  Department  of  Defense,  it  is  essential  that  the 
LSS  methodology  be  suitably  modified  in  its  implementation  in  the  military.  We  discuss  below  a 
set  of  issues  that  must  be  addressed  in  implementing  LSS  in  the  military. 

Experience  indicates  that  the  success  of  Lean  Six  Sigma  depends  on  employee 
empowerment  and  participative  management.  Since  the  military  is  traditionally  organized  and 
managed  as  a  strict  hierarchy,  implementing  LSS  is  a  challenging  task.  Also,  frequent  rotation 
and  movement  of  officers  in  their  assignments  is  a  common  practice  in  the  military.  This 
creates  a  possibility  that  the  procedures  and  culture  created  by  one  officer  in  implementing  LSS 
can  be  disrupted  when  s/he  is  replaced  by  another  officer. 

In  the  military,  the  employees  may  enthusiastically  embrace  LSS  implementation  initially, 
but  it  is  difficult  to  maintain  that  enthusiasm  towards  LSS  in  the  long-run  without  proper  incentive 
systems.  Private-sector  organizations  can  give  financial  incentives  to  employees  to  reward  their 
contributions  to  process-improvement  efforts.  However,  it  is  almost  impossible  to  give  such 
monetary  incentives  in  the  military  due  to  the  governmental  rules  and  regulations.  Hence,  an 
alternate  non-monetary  incentive  system,  for  example,  for  career  enhancement  or  for  better 
promotion  opportunities,  must  be  investigated. 

Another  area  to  be  carefully  studied  regarding  implementing  LSS  in  the  military  is  to 
understand  the  fundamental  nature  of  military  operations.  Lean  Six  Sigma  methodologies  were 
originally  designed  for  manufacturing  assembly  systems  in  which  the  demands  are  known  or 
predetermined.  As  we  move  closer  to  a  foxhole  from  a  factory ,  the  overall  magnitude  of 
uncertainty  in  demand,  supply,  and  environment  increases  significantly.  Military  planners  must 
fully  keep  in  mind  that  the  demand  and  supply  are  uncertain  in  many  military  applications; 
hence,  LSS  must  be  selectively  implemented  in  different  parts  of  the  military  in  different  ways. 

For  example,  supply  officers  may  be  encouraged,  but  not  required,  to  apply  Lean  and  just-in- 
time  concepts  to  reduce  inventory  in  military  operational  environments;  such  should  not  be 
required  due  to  the  inherent  nature  of  uncertain  demand  and  the  potentially  heavy  penalty  of 
stockouts  that  would  cause  readiness  degradation  and  potential  losses  of  human  lives. 

Finally,  we  wish  to  point  out  that  while  the  issues  identified  above  are  important  and  must  be 
carefully  analyzed  by  military  planners,  approaches  for  dealing  with  them  can  be  developed.  Moreover, 
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the  benefits  of  reduced  lifecycle  costs  and  improved  readiness  that  can  be  realized  from  implementing 
Lean  Six  Sigma  are  simply  too  great.  Hence,  we  believe  that  implementing  Lean  Six  Sigma  in  the 
military  is  a  strategically  important  logistics  initiative  and  recommend  that  it  be  undertaken  under  full 
steam. 
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Abstract 

Aircraft  Intermediate  Maintenance  Division  at  Naval  Air  Station  Lemoore  achieved  time 
and  cost  reductions  using  the  NAVAIR  Enterprise  AIRSpeed  program  of  Lean,  Six  Sigma  and 
Theory  of  Constraints,  but  could  changes  in  organization  structure  or  management  practices 
provide  further  improvements? 

Organizational  simulation  software  was  employed  to  test  interventions  that  could  reduce 
throughput  time  for  the  F414  aircraft  engine.  A  baseline  model  was  developed,  and 
interventions  were  modeled  and  simulated.  The  simulated  results  indicated  that  paralleling 
some  tasks  could  significantly  decrease  maintenance  duration  while  maintaining  quality.  The 
intervention  was  implemented — saving  26  days  per  engine.  Organizational  modeling  and 
simulation  can  identify  and  pre-test  time  and  cost  savings  over  and  above  techniques  such  as 
Lean  and  Six  Sigma. 

Key  words:  Organizational  modeling,  organizational  simulation,  Logistics,  Lean,  Six 

Sigma 

Introduction 

The  Aircraft  Intermediate  Maintenance  Division  (AIMD)  at  Naval  Air  Station  (NAS) 
Lemoore,  CA,  has  worked  aggressively  to  reduce  engine  maintenance  time  using  the  tools  of 
the  NAVAIR  Enterprise  AIRSpeed  (AIRSpeed)  program.  AIRSpeed  is  a  system  created  by  the 
Naval  Aviation  Readiness  Integrated  Improvement  Program’s  (NAVRIIP)  to  enable  cost-wise 
readiness  across  the  naval  aviation  enterprise  (Naval  Air  Forces  Public  Affairs  Office,  2006). 
AIMD  Lemoore  has  achieved  time  and  cost  reductions  at  the  maintenance  activity  level  using 
AIRSpeed’s  prescribed  tools  of  Theory  of  Constraints,  Lean,  and  Six  Sigma,  but  could  further 
improvements  be  made  by  changing  the  organization  structure  or  management  practices? 

In  an  effort  to  answer  this  research  question,  AIMD  Lemoore  teamed  with  the  Naval 
Postgraduate  School’s  (NPS)  Graduate  School  of  Business  and  Public  Policy  to  explore 
organizational  modeling  as  a  method  for  identifying  potential  modifications  to  the  organization 
which  may  improve  AIMD  performance.  Specifically,  AIMD  leadership  focused  on  improving 
F414  aircraft  engine  maintenance  by  decreasing  engine  throughput  duration. 

The  objective  of  this  effort  was  to  provide  the  NAS  Lemoore  AIMD  400  Division,  the 
Division  responsible  for  F414  maintenance,  with  recommendations  on  how  their  organization 
may  be  restructured  in  order  to  decrease  F414  maintenance  cycle-time.  To  meet  this  objective, 
NPS  developed  an  organizational  model  of  the  400  Division  which  described  their  current  F414 
maintenance  process.  This  model  was  then  modified  to  characterize  the  impact  of 
organizational  changes  on  maintenance  cycle-time. 

This  paper  is  organized  into  four  sections.  The  first,  a  literature  search,  provides  a  basis 
for  understanding  organizational  modeling  in  general  as  well  as  techniques  specific  to  the 
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POWer  software  developed  by  Dr.  Raymond  E.  Levitt’s  Virtual  Design  Team  (VDT)  research 
group  at  Stanford  University  and  employed  in  this  project.  The  second  section  discusses  the 
methodology  for  conducting  this  study.  The  third  section  presents  the  results  of  the  modeling 
effort.  Finally,  section  four  presents  project  conclusions,  recommendations  for  restructuring  the 
400  Division,  and  recommendations  for  future  research. 

Computational  Organizational  Modeling 

In  the  21st  Century,  computational  organizational  modeling,  a  new  predictive  modeling 
technique,  has  come  of  age.  It  is  a  tool  which  has  the  potential  to  assess  how  changes  to  an 
organization  may  or  may  not  benefit  the  organization’s  performance  (Levitt,  2004). 
Computational  organizational  modeling  as  a  tool  for  improving  quality  is  different  from  many 
other  quality-improvement  techniques  such  as  Lean,  TOC,  or  Six  Sigma  in  that  it  does  not  focus 
on  the  production  process,  but  instead  on  the  organizational  structure  that  manages  that 
production  process,  and  the  information  flow  through  that  organization  necessary  to  execute  the 
production  process.  It  is  based  upon  the  understanding  that  by  improving  the  quality  of  the 
organization  and  the  flow  of  information  through  that  organization,  the  quality  of  the 
organization’s  output  can  be  improved. 

The  technique  of  organizational  modeling  is  analogous  to  modeling  employed  in  the 
natural  sciences,  such  as  finite  element  modeling  (FEM)  or  computational  fluid  dynamics  (CFD) 
modeling.  FEM  and  CFD  modeling  both  break  down  the  larger  structure  being  modeled  into 
smaller  elements,  with  each  element  having  its  own  characteristics:  e.g.,  modulus  of  elasticity, 
density,  viscosity.  With  an  understanding  of  how  these  elements  interact,  the  overall  effect  of  a 
force  or  moment  on  the  larger  structure  can  be  assessed.  Similarly,  organizational  modeling  is 
accomplished  by  breaking  down  an  organization  into  smaller  elements  such  as  tasks,  people, 
and  communication  methods — each  with  their  own  characteristics,  e.g.,  time  required  to 
accomplish  a  task,  worker  experience,  communication  clarity — and  predicting  how  changes  to 
an  organization  may  affect  each  element  and,  subsequently,  how  those  elements  in  turn  affect 
the  overall  organizational  performance  (Levitt,  2004). 

This  detailed  level  of  organizational  characterization  theoretically  allows  managers  to 
design  their  organization  in  the  same  way  that  engineers  design  bridges.  Organizational 
modeling  allows  managers  to  perform  “what-if”  analyses,  evaluating,  in  a  virtual  environment, 
the  effects  of  organizational  constructs  in  order  to  identify  the  structure  resulting  in  the  best 
output.  Gaining  similar  insight  without  the  aid  of  a  modeling  tool  would  be  prohibitive. 
Organizations  could  not  withstand  the  dynamics  of  change  after  change  simply  to  determine 
what  works  best  and  what  does  not. 

The  organizational  model  employed  in  this  project  is  POWer,  version  1.1.6.  It  was 
developed  by  Dr.  Raymond  Levitt  as  part  of  a  suite  of  Virtual  Design  Team  (VDT)  simulations  at 
Stanford  University. 

Virtual  Design  Team — POWer 

POWer  evolved  from  the  Virtual  Design  Team  simulations,  which  are  based  on  macro¬ 
contingency  theory  and  describe  work  in  terms  of  information  flow  (Thomsen,  Kunz,  Levitt  & 
Nass,  1998).  POWer  is  based  on  the  premise  that  no  matter  what  business  an  organization  is 
in,  be  it  production  of  widgets,  design  of  skyscrapers,  or  providing  hotel  rooms,  one  thing  they 
all  have  in  common  is  they  must  process  information  effectively  to  do  their  job  well  (Kunz, 

1998). 
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Theoretical  Basis  for  POWer 

The  concept  that  organizations  can  be  modeled  in  terms  of  information  flow  is  based  on 
J.R.  Galbraith’s  theory  of  information  processing.  According  to  Galbraith,  information  transfer 
and  processing  is  dynamic.  Due  to  the  complexity  of  information  and,  many  times,  the  sheer 
amount  of  it,  there  are  often  instances  in  which  an  individual  is  unable  to  process  all  of  the 
information  he  is  given  because  he  does  not  have  the  skill  or  experience  to  make  decisions 
quickly  enough.  As  a  result,  an  exception  (as  Galbraith  defines  it)  is  created.  Exceptions  are 
common  in  today’s  fast-paced  world  in  which  we  are  inundated  with  requests  from  e-mail,  cell 
phones,  black-berries,  etc.  In  Galbraith’s  view,  organizations  are  modeled  primarily  as 
hierarchies;  it’s  through  these  hierarchies  that  exceptions  are  passed  up  the  “chain  of 
command”  to  be  handled  by  more  experienced  individuals.  Along  with  the  hierarchical  structure 
by  which  exceptions  are  passed,  Galbraith  notes  there  are  also  exchanges  of  information 
between  individuals  at  equal  levels  in  an  organization.  These  information  exchanges  can  also 
be  used  to  handle  exceptions,  and  are  often  more  effective  than  those  moving  up  the  chain  of 
command  since  they  tend  less  to  overload  upper-level  managers  and  create  additional 
exceptions  (Thomsen,  Kunz,  Levitt  &  Nass,  1998). 

Methodology 

The  researchers  visited  the  NAS  Lemoore  in  order  to  conduct  multiple  interviews  with 
400  Division  personnel.  Information  was  collected  to  properly  structure  the  400  Division  model 
in  POWer  and  accurately  characterize  the  properties  of  each  software  element.  Through  these 
exchanges,  a  baseline  model  was  created  that  accurately  characterized  the  operation  of  the  400 
Division  F414  maintenance  process. 

Modifications,  also  termed  “interventions,”  were  identified  which  have  potential  for 
decreasing  F414  maintenance  throughput.  Each  intervention  was  separately  created  by 
modifying  the  baseline  model.  Comparisons  between  these  modified  and  baseline  models  were 
made  to  determine  the  utility  of  each  intervention.  Finally,  a  combined  intervention  model 
incorporating  all  individual  interventions  deemed  beneficial  was  developed  and  compared  to  the 
baseline  model. 

Figure  1  presents  the  baseline  model  of  the  400  Division.  The  slanted  boxes  at  the  top 
of  the  figure  represent  meetings.  The  human-shaped  symbols  represent  positions  within  the 
division.  The  boxes  in  the  center  of  the  figure  represent  the  primary  F414  maintenance  tasks, 
while  the  boxes  in  a  vertical  line  on  the  left  represent  the  off-core  tasks.  The  remaining 
polygons  represent  milestones  in  the  maintenance  process. 
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Figure  1.  Organizational  Model 


The  positions  modeled  were  those  that  directly  impacted  F414  maintenance.  Positions 
were  modeled  in  terms  of  the  number  of  personnel  assigned,  amount  of  time  available  to  work 
F414  tasks,  qualifications,  skill  levels,  and  experience.  Since  this  engine  was  one  of  the  six 
engines  the  division  maintained,  the  time  as  allocated  was  1/6  of  the  actual  time  available.  In 
addition,  off-core  tasks  described  below  were  added  to  a  positions’  workload  to  occupy  their 
time  when  not  conducting  F414  maintenance.  Figure  2  presents  the  organizational  structure. 
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Figure  2.  NAS  Lemoore  AIMD  400  Division  Information  Hierarchy 

The  terminology  used  in  Figure  2  and  throughout  this  report  to  reference  individuals  and 
groups  is  consistent  with  terminology  used  in  the  Navy’s  AIMD.  For  clarity,  these  terms  are 
defined  as  follows: 

Div-0:  Division  Officer 

PC  Officer:  Production  Control  Officer 

AZ:  administrative  personnel 

41V:  personnel  who  directly  conduct  F414  maintenance 

05E:  supply  personnel  dedicated  to  the  division 

450:  personnel  responsible  for  conducting  final  tests  of  the  F414 

LPO:  Leading  Petty  Officer,  responsible  for  the  workcenter 

Tasks  were  modeled  in  terms  of  duration,  required  skills,  priority,  and  complexity. 
Modeled  tasks  are  presented  in  Figure  3.  The  following  is  a  general  description  of  the  F414 
maintenance  process. 

After  the  engine  is  received,  AZ  personnel  begin  by  comparing  information  in  the  engine 
logbook  to  information  in  two  central  databases,  AEMS  and  SAME,  which  track  engine  parts 
and  engine  movement.  Prior  to  maintenance  action  commencing  on  the  engine,  AZ  personnel 
must  resolve  any  discrepancies.  Once  completed,  41V  personnel  conduct  a  major  engine 
inspection  (MEI)  followed  by  an  engine  teardown  to  determine  which  engine  modules  need 
replacing.  Replacement  modules  are  pulled  from  supply  by  05E  personnel.  The  engine  is 
reassembled  or  “built-up”  by  41V  personnel,  and  then  sent  to  the  test  cell,  where  450  personnel 
run  it  through  pre-defined  profiles  assessing  operability.  The  engine  is  returned  to  the 
maintenance  hanger  where  41V  personnel  conduct  a  post-test  inspection.  At  this  point,  AZ 
personnel  complete  paperwork;  Controller  personnel  sign  off  the  engine  as  ready  for  issue  (RFI) 
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to  an  operational  squadron.  Throughout  this  process,  Controllers  are  directing  the  maintenance 
activities. 


Figure  3.  Work  Breakdown  Structure  of  the  F414  Engine  Maintenance  Process 


To  ensure  positions  were  continually  occupied  throughout  the  F414  maintenance 
process,  as  they  would  be  in  reality,  off-core  tasks  were  added  to  the  model  to  simulate 
maintenance  work  being  accomplished  by  personnel,  other  than  maintenance  of  the  single 
engine  being  modeled. 


Meetings  were  modeled  in  terms  of  duration,  who  attended,  priority  and  interval  time 
between  meetings.  Meetings  were  a  key  method  of  reliably  transferring  information  between 
personnel.  In  general,  the  division  had  a  set  of  morning  meetings  and  a  set  of  afternoon 
meetings. 

Rework  was  modeled  as  a  percentage  of  work  accomplished.  Most  F414  rework 
occurred  at  the  test-cell  phase  of  maintenance.  The  percentage  of  rework  was  based  on  400 
Division  estimates. 


Additional  organizational  characteristics  modeled  included  the  overall  experience  level  of 
the  division,  the  degree  of  centralized  control,  the  degree  of  formality  in  transferring  information 
(i.e.,  meetings  versus  hallway  conversation),  and  the  matrix  strength  or  connectedness  of 
personnel. 


Model  Validation  Procedure 

Once  the  model  was  constructed,  the  maintenance  duration  predicted  by  the  model  was 
compared  to  the  actual  time  it  should  take  to  conduct  engine  maintenance.  The  actual  time  was 
calculated  by  summing  the  duration  of  all  tasks  occurring  in  series  and  the  longest  duration  task 
of  any  grouping  of  tasks  occurring  in  parallel.  The  smaller  the  difference  was  between  these 


LUSUsiuriiKuvri™ 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -621  - 


values,  the  higher  the  confidence  in  the  model,  and,  hence,  the  predicted  impacts  of 
interventions. 

Model  Interventions 

Once  the  model  was  determined  to  accurately  depict  the  current  organization, 
modifications  or  interventions  were  made  to  evaluate  alternate  organizational  constructs  which 
might  reduce  throughput  duration.  The  following  interventions  to  the  baseline  model  were 
evaluated. 

1 .  Parallel  AZ  Acceptance  task  with  other  maintenance  tasks 

2.  Combine  AZ  and  Controller  positions 

3.  Combine  41 V  and  450  positions 

4.  Decrease  organization’s  centralization 

5.  Add  additional  personnel  to  each  position 

6.  Alter  current  meetings’  duration  and  frequency 

7.  Combine  meetings 

8.  Conduct  a  combined  intervention 

The  current  F414  maintenance  process  presented  in  Figure  3  shows  a  serial  process 
initiated  by  the  AZ  Acceptance  tasks. 

Intervention  1 :  Considered  the  impact  of  conducting  the  AZ  Acceptance  tasks  in  parallel 
with  all  other  maintenance  tasks. 

Intervention  2:  Personnel  assigned  to  the  AZ  and  Controller  positions  are  combined  into 
a  single  position.  This  position  is  assigned  the  combination  of  tasks  originally  assigned  to  the 
separate  positions.  This  intervention  was  evaluated  in  two  sub-interventions,  first  without 
retraining  individuals  and  then  with  retraining. 

Intervention  3:  The  same  as  Intervention  2,  with  the  work  positions  combined. 

Intervention  4:  One  of  the  impacts  of  AIRSpeed  is  to  decrease  the  centralized  control  of 
an  organization  by  pushing  authority  for  decision-making  to  the  lowest  possible  level.  This 
fourth  intervention  assessed  the  impact  of  the  division’s  further  decreasing  centralization. 

Intervention  5:  Assessed  the  impact  of  adding  additional  personnel  to  existing  positions. 
Personnel  were  added  separately  to  AZ,  Controller,  41V  Crew,  05E  Crew,  and  450  Crew 
positions  while  holding  personnel  at  all  other  position  constant. 

Intervention  6:  Considering  maintenance  tasks  are  well  defined  and  the  personnel  are 
highly  skilled,  it’s  conceivable  that  that  altering  meeting  duration  and/or  frequency  may  decrease 
F414  throughput  duration.  This  intervention  evaluated  altering  the  duration  and  frequency  of  the 
0700  morning  meeting,  the  Division’s  primary  coordination  meeting. 

Intervention  7:  For  the  same  rationale  as  intervention  #6,  this  intervention  evaluated  the 
impact  of  combining  all  of  the  morning  meetings  while  leaving  the  afternoon  meetings  separate. 
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It  then  evaluated  the  impact  of  separately  combining  all  morning  meetings  and  all  afternoon 
meetings. 

Intervention  8:  Based  on  the  results  of  the  single  interventions,  a  combined  intervention 
was  developed  which  included  those  interventions  presented  above  that  decreased  the  F414 
maintenance  throughput  time. 

Evaluating  Interventions 

Interventions  were  evaluated  by  comparing  four  metrics  predicted  by  the  baseline  model 
to  those  predicted  by  the  models  with  interventions.  The  first  metric  was  project  duration,  the 
duration  required  to  accomplish  maintenance  of  a  single  F414.  Duration  was  considered  the 
most  important  metric.  The  second  metric  was  position  backlog,  a  measure  of  the  number  of 
days  of  work  a  position  has  yet  to  accomplish.  It  is  analogous  to  the  size  of  a  person’s  in-box. 

A  position  with  a  high  backlog  poses  a  risk  of  increasing  project  duration  and  decreasing  output 
quality.  Position  backlog  is  presented  as  a  line  graph  of  number  of  backlog  days  versus  time. 
The  third  metric  was  cost.  Although  absolute  cost  was  not  a  concern  for  this  study,  changes  in 
costs  resulting  from  interventions  were.  Of  particular  interest  were  interventions  resulting  in 
increases  in  costs  associated  with  the  major  tasks  of  engine  teardown,  buildup,  and  testing. 

Cost  was  calculated  by  the  simulation  and  output  in  both  text  and  graphic.  The  fourth  metric  was 
functional  risk,  the  risk  that  an  engine  has  defects  due  to  rework  and  the  inability  of  personnel  to 
handle  problems.  Qualitative  comparisons  of  functional  risk  were  made  using  output  charts  of 
the  functional  risk. 

For  any  given  intervention,  the  impact  on  each  of  the  four  metrics  was  categorized  as 
positive,  negative,  or  no  impact  and  given  a  rating  respectively.  For  example,  a  decrease  in 
project  duration  resulting  from  an  intervention  would  be  considered  positive,  while  an  increase  in 
cost  or  risk  would  be  considered  negative 


Results 

The  Results  section  begins  with  a  presentation  of  the  baseline  model  validation  results. 
The  baseline  model  is  followed  by  a  summary  of  the  results  of  the  seven  individual  interventions 
and  the  combined  intervention.  Finally,  there  is  a  discussion  of  which  interventions  were 
implemented  and  their  impact  on  F414  maintenance  duration. 

Baseline  Model  Evaluation 

The  actual  time  required  to  conduct  F414  maintenance  was  calculated  to  be  21.77 
days — compared  to  the  baseline  model  prediction  of  21 .09  days.  Since  these  two  durations 
were  within  3%  of  each  other,  there  was  high  confidence  that  the  baseline  model  was  accurate. 

Interventions — Summary  of  Results 

Table  1  presents  a  summary  of  the  intervention  results.  The  first  intervention,  paralleling 
the  AZ  Acceptance  Task,  has  the  greatest  benefit  on  decreasing  F414  throughput  duration. 
Other  interventions  that  were  beneficial  included  decreasing  centralization,  and  separately 
combining  the  morning  and  afternoon  meetings.  The  combined  intervention,  incorporating  all  of 
these  beneficial  interventions,  resulted  in  a  35%  decrease  in  F414  throughput  duration  with  a 
slight  decreasing  in  the  backlog  of  most  of  the  personnel.  A  detailed  discussion  of  the  analysis 
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and  results  associated  with  Intervention  1  is  presented  in  the  following  sub-section.  All  other 
interventions  (including  the  combined  intervention)  were  analyzed  in  the  same  manner. 


Intervention 

Project  Duration 

Backlog 

Cost 

Risk 

1.  Parallel  AZ 
Acceptance 

58.56  hour 
decrease 

Decrease  most 
positions 

No  significant  impact 

Increase  in  AZ 
Acceptance  task 
risk 

2a.  Combine 
Controller  &  AZ 
positions  without 
training 

110  hour  increase 

Decrease 
controller  &  AZ, 
Increase  for  Div- 
O  &  PC 

AZ  Acceptance  task 
work  &  rework  cost 
increase  by  205.6  & 
11.72  respectively 

Increase  in  AZ 
Acceptance  task 
risk 

2b.  Combine 
Controller  &  AZ 
positions  with 
training 

56.7  hour 
increase 

Decrease  in 
Controller  &  AZ 
backlog. 

Increase  for  Div- 
O  &  PC 

AZ  Acceptance  task 
work  &  rework  cost 
increase  by  140.1  &  18 
respectively 

Increase  in  AZ 
Acceptance  task 
risk 

3a.  Combine  41V 
and  450  positions 
without  training 

132.6  hour 
increase 

Slight  decrease 
in  41V  and  450 
backlog 

Increase  costs: 

Buildup  &  rework — 
267.16  and  7.2,  Test 
work,  rework,  and  wait 
costs — 1085,  61.5, 

290.2 

3/4  top  risk  areas 
assigned  to 
combined  41V- 
450  vs  2/4 
currently 

3b.  Combine  41V 
and  450  positions 
without  training 

67.6  hour 
increase 

Slight  decrease 
in  41V  and  450 
backlog 

Increase  costs: 

Buildup  work  -  267.15 
&  test  work,  rework, 
and  wait  costs  -  303.4, 
5.63,  93.41 

3/4  top  risk  areas 
assigned  to 
combined  41V- 
450  vs  2/4 
currently 

4.  Decrease 
Centralization 

4.4  hour 
decrease 

No  significant 
impact 

Slight  increase  in 

Buildup  task  rework 
costs  of  9.86 

No  significant 
impact 

5a.  Add  AZ 
personnel 

1 .87  min  saved  / 
person 

No  data 
collected 

No  data  collected 

No  significant 
impact 

5b.  Add  Controller 
personnel 

6.82  min  lost  / 
person 

No  data 
collected 

No  data  collected 

No  significant 
impact 

5c.  Add  41V  Crew 
personnel 

0.91  min  lost  / 
person 

No  data 
collected 

No  data  collected 

No  significant 
impact 

5d.  Add  05E  Crew 
personnel 

10.51  min  saved  / 
person 

No  data 
collected 

No  data  collected 

No  significant 
impact 

5e.  Add  450  Crew 
personnel 

4.42  min  saved  / 
person 

No  data 
collected 

No  data  collected 

No  significant 
impact 

6a.  Vary  0700 
meeting  duration  & 
frequency 

6.56  hours  saved 
due  to  less 
frequent  meeting 

No  data 
collected 

No  data  collected 

No  significant 
impact 

6b.  Vary  0630 
meeting  frequency 

1.6  hours  saved 
due  to  less 
frequent  meetings 

No  data 
collected 

No  data  collected 

Slight  increase  in 
risk  when 
increasing  time 
between 
meetings 

7a.  Combine  only 
morning  meetings 

No  significant 
impact 

No  data 
collected 

No  data  collected 

No  significant 
impact 
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7b.  Separately 
combine  morning 
and  end  of  day 
meetings 

7.28  hours  saved 
by  decreasing 
meeting 
frequency  to 
every  other  day 

No  data 
collected 

No  data  collected 

No  significant 
impact 

8.  Combined 
Interventions 

58.96  hours 
saved 

Decreases  most 
positions. 
Increases  450 
LPO 

Buildup  rework 
decreases  by  26.3; 
Teardown  rework 
increases  by  10.49 

No  significant 
impact 

Table  1.  Simulation  Results  for  Interventions  by  Duration,  Backlog,  Cost  and  Risk 

Intervention  1,  paralleling  the  acceptance  task  with  maintenance,  decreased  project 
duration  by  7  days  from  the  base  model  prediction  of  21 .09  days  to  13.77  days.  The  impact  of 
this  intervention  on  individual  task  durations  and  the  overall  duration  are  depicted  in  Figure  4. 
The  dark  bars  represent  the  duration  of  the  individual  maintenance  tasks — with  the  exception  of 
the  second  through  eleventh  bars,  which  represent  off-core  tasks.  The  decrease  in 
maintenance  duration  is  the  result  of  starting  on-engine  maintenance  tasks  at  the  same  time  as 
the  AZ  Acceptance  task.  In  the  current  process  (Figure  4  left  chart),  the  14-day  AZ  Acceptance 
task  must  be  accomplished  before  any  other  tasks.  This  intervention  (depicted  in  the  right  chart 
in  Figure  4)  allows  the  engine-maintenance-related  tasks  to  begin  at  the  same  time  as  the  AZ 
Acceptance  task. 
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Figure  4.  Comparison  of  Project  Duration  for  Intervention  1 

Figure  5  presents  a  comparison  of  the  position  backlog  of  the  baseline  model  and  the 
model  employing  the  intervention.  The  comparison  shows  a  slight  decrease  in  position  backlog 
resulting  from  Intervention  1. 
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Figure  5.  Comparison  of  Position  Backlog  for  Intervention  1 

Figure  6  presents  a  comparison  between  baseline  model  costs  associated  with  the 
model  employing  the  intervention.  Figure  6  indicates  no  significant  impact  on  cost. 


Current  Process 
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Figure  6.  Comparison  of  Cost  for  Intervention  1 

Figure  7  presents  a  comparison  of  the  baseline  model  functional  risk  with  the  functional 
risk  for  the  model  implementing  Intervention  1 .  This  comparison  indicates  a  slight  risk  increase 
for  the  AZ  Acceptance  task.  This  is  not  unexpected  since  this  task  is  now  being  accomplished 
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in  conjunction  with  other  tasks;  hence,  the  time  originally  devoted  by  Controller  personnel  to 
assist  AZ  personnel  in  handling  problems  must  now  be  devoted  not  only  to  AZ  personnel,  but  to 
other  personnel  concurrently  working  engine-maintenance  tasks. 
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Figure  7.  Comparison  of  Functional  Risk  for  Intervention  1 

The  overall  rating  for  Intervention  1  is  presented  in  Table  1.  This  intervention  was 
considered  beneficial  because  it  resulted  in  a  significant  decrease  in  project  duration,  a  slight 
decrease  in  position  backlog,  no  significant  impact  on  cost,  and  only  a  slight  increase  in 
functional  risk  for  a  single  task. 

Intervention  #8 — Combined  Intervention 

The  combined  intervention  included  the  following  interventions,  which  were  chosen  for 
being  the  most  beneficial: 

Intervention  #1 — Paralleling  the  acceptance  task 

Intervention  #4 — Decreasing  centralization  from  high  to  low 

Intervention  #6 — Decreasing  0700  meeting  frequency  to  every  2  days 

Intervention  #7 — Separately  combining  morning  &  afternoon  meetings 

The  impact  of  these  combined  interventions  on  project  duration  was  a  decrease  from 
21.09  days  to  13.72  days.  The  backlog  for  most  positions  decreased  with  an  increase  in  only 
one  position,  the  450  LPO.  There  was  a  slight  increase  in  Teardown  task  rework  cost  from 
$26.44  to  $36.93,  and  a  slight  decrease  in  the  Buildup  task  rework  cost  from  $48.43  to  $22.13. 
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Overall,  the  changes  in  cost  were  not  considered  significant.  Finally,  there  was  no  significant 
impact  on  functional  risk. 

Assessment  of  interventions 

The  results  of  this  study  lead  to  the  conclusion  that  four  of  the  seven  interventions  to  the 
division  considered  in  this  study  would  be  beneficial  to  reducing  the  throughput  duration: 
paralleling  the  AZ  Acceptance  task,  decreasing  centralization,  decreasing  0700  meeting 
frequency,  and  separately  combining  morning  and  afternoon  meetings. 

The  greatest  benefit  to  reducing  the  F414  throughput  duration  comes  from  paralleling 
the  AZ  Acceptance  task.  Although  this  intervention  increases  functional  risk,  this  increase  is 
minor  relative  to  the  decrease  in  throughput  time  by  7.21  days.  There  is  also  a  consequent 
decrease  in  position  backlog. 

Decreasing  centralization,  a  benefit  realized  through  the  implementation  of  AIRSpeed, 
also  has  a  positive  impact  on  decreasing  F414  throughput.  This  intervention  resulted  in  a  4.4- 
hour  decrease  in  duration. 

By  decreasing  the  0700  meeting  frequency  from  every  day  to  every  other  day,  F414 
throughput  duration  decreases  by  6.56  hours.  This  benefit  is  the  result  of  a  highly  skilled 
workforce  executing  well-defined  tasks,  allowing  personnel  to  spend  more  time  working  on 
engine  maintenance  and  less  time  exchanging  information  in  meetings. 

By  separately  combining  morning  and  afternoon  meetings  such  that  there  is  one 
morning  meeting  and  one  afternoon  meeting  that  all  personnel  attend,  F414  throughput  duration 
decreases  by  7.28  days.  At  the  same  time,  there  is  also  no  increase  in  functional  risk. 

Unfortunately,  benefits  associated  with  combining  these  four  interventions  are  not 
additive.  This  makes  sense  based  on  their  interrelated  nature.  When  combining  interventions, 
the  benefit  to  reducing  F414  throughput  duration  is  nonetheless  significant  in  that  there  is  a 
reduction  of  over  35%  from  the  baseline  case  representing  the  current  organization.  In 
conjunction  with  this  benefit,  there  is  a  decrease  in  backlog  for  all  positions  excluding  one,  the 
450  LPO,  and  there  is  no  adverse  impact  to  cost  or  functional  risk. 

Two  other  interventions  considered,  combining  the  AZ  and  Controller  positions  and 
combining  the  41V  and  450  positions,  resulted  in  increases  in  F414  throughput  duration  as  well 
as  increases  in  cost  and  risk,  with  the  only  predicted  benefit  being  a  decrease  in  position 
backlog  for  the  combined  positions.  Clearly,  these  interventions  are  not  beneficial. 

Finally,  the  intervention  associated  with  adding  additional  personnel  did  not  affect  F414 
throughput  duration,  and  had  no  impact  on  risk.  Obviously,  there  would  be  no  benefit  to 
implementing  this  intervention. 

Recommended  Interventions 

■  NPS  recommended  the  400  Division  implement  the  following  interventions 

■  Decrease  0700  morning  meeting  frequency  to  every  other  day. 
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■  Separately  combine  morning  and  end-of-day  meetings 

■  Parallel  the  AZ  Acceptance  task 

The  first  and  each  subsequent  intervention  recommendation  should  be  implemented, 
followed  by  a  period  of  evaluation.  The  priority  order  of  these  interventions  is  based  on  first 
implementing  those  interventions  that  can  most  easily  be  reversed.  For  example,  conducting 
the  0700  meeting  every  other  day  is  a  relatively  easy  organizational  change  which  should  result 
in  a  decrease  in  F414  throughput  duration.  At  the  same  time,  it  is  an  organization  change  that 
can  be  reversed  if  deemed  necessary. 

Impact  of  implementing  interventions 

The  NAS  Lemoore  AIMD  and  400  Division  leadership  had  significant  confidence  in  the 
results  of  this  study,  and  chose  to  fully  implement  the  recommendation  to  parallel  the  AZ 
acceptance  task  while  partially  implementing  the  recommendation  to  separately  combine  the 
morning  and  afternoon  meetings.  The  impacts  of  these  decisions  were  quickly  realized  and 
deemed  successful.  The  following  discussion  presents  three  instances  in  which  paralleling  the 
AZ  acceptance  task  significantly  reduced  F414  maintenance  throughput  time.  Table  2  at  the 
end  of  this  section  presents  a  summary  of  these  results.  Following  this  is  a  discussion  of  how 
partially  combining  400  Division  morning  meetings  improved  organizational  performance. 

On  20  October,  the  400  Division  received  F414  serial  number  868472  from  VFA106, 
NAS  Oceana.  On  that  same  day,  the  engine-acceptance  process  commenced.  During  the 
acceptance  process,  SAME  database  problems  were  identified.  Recall  that  the  SAME 
database,  described  earlier  in  this  paper,  is  an  historical  record  of  maintenance  actions 
accomplished  on  each  engine.  Often,  an  engine  is  received  by  the  400  Division  for  which  there 
are  discrepancies  between  data  contained  in  the  SAME  database  and  the  engine  log  book. 
These  SAME  discrepancies  were  resolved  on  7  November.  Prior  to  implementing  the 
intervention  of  paralleling  the  AZ  acceptance  process,  teardown  would  not  have  started  until 
after  the  SAME  database  problems  were  resolved  on  7  November.  By  implementing  this 
intervention,  however,  engine  maintenance  began  on  23  October  when  personnel  were 
available — saving  16  days,  the  difference  between  23  October  versus  7  November.  (See  Table 
2.) 


In  the  second  observation,  on  25  October,  the  400  Division  received  F414  serial  number 
868083  from  VFA-2.  SAME  database  problems  were  identified  on  26  October  which  were 
resolved  on  13  November.  By  choosing  to  implement  the  intervention  of  paralleling  the  AZ 
acceptance  process,  maintenance  on  this  engine  commenced  on  29  October  versus  waiting 
until  13  November,  thus  saving  16  days. 

Finally,  on  5  September,  the  400  Division  received  F414  engine  serial  number  868265 
from  the  USS  Lincoln.  On  that  same  day,  SAME  database  problems  were  identified  which  were 
eventually  resolved  on  16  October.  A  total  of  46  days  were  saved  in  this  case  by  paralleling  the 
AZ  acceptance  process  since  maintenance  on  this  engine  started  on  6  September.  (See  Table 
2.) 
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Engine  Serial 
Number 

Engine 

Received 

SAME 

Problem(s) 

Identified 

SAME 

Problem(s) 

Resolved 

Engine 

Maintenance 

Started 

Days  Saved 

868472 

VFA-106 

20  Oct 

20  Oct 

7  Nov 

23  Oct 

16 

868083 

VFA-2 

25  Oct 

26  Oct  06 

13  Nov 

29  Oct 

16 

868265 

USS  Lincoln 

5  Sep 

5  Sep 

16  Oct 

6  Sep 

46 

Table  2.  Summary  of  Intervention  Results 

Like  the  impacts  presented  in  Table  2,  the  AIMD  and  400  Division  leadership’s  decision 
to  combine  certain  aspects  of  their  morning  meetings  also  had  a  positive  impact  on  decreasing 
the  time  required  to  conduct  F414  maintenance.  Specifically,  LPO  coordination  efforts 
conducted  at  both  the  0630  and  0700  meetings  were  combined.  At  the  same  time,  the  duration 
spent  by  each  LPO  in  this  combined  meeting  was  decreased,  allowing  them  to  more  quickly 
provide  direction  to  their  subordinates. 

At  the  time  of  this  article’s  writing,  this  intervention  had  just  recently  been  implemented, 
and  quantitative  results  of  its  impact  were  not  yet  available.  Qualitatively,  though,  the  Division 
Officer  in  charge  of  the  400  Division  has  identified  a  marked  improvement  in  the  amount  of  work 
being  accomplished  as  a  result  of  implementing  this  intervention.  Prior  to  its  implementation, 
upon  his  arrival  to  the  office  at  0630  each  day,  the  Div-0  would  see  a  significant  amount  of 
coordination  work  being  accomplished  by  LPO  and  PC  personnel  in  preparation  for  the  day’s 
work.  Following  the  combination  of  morning  meetings,  the  Div-0  arrives  at  work  and  now  sees 
personnel  working  on  the  F414  engines.  Information  flow  is  being  accomplished  more 
smoothly,  thus  allowing  coordination  efforts  to  be  accomplished  more  quickly;  hence,  more  work 
is  accomplished  on  a  given  day. 

The  AIMD  and  400  Division  leadership  are  pleased  with  the  results  of  these 
interventions.  Both  quantitatively  and  qualitatively,  their  impacts  have  resulted  in  shorter  F414 
throughput  time  and  improved  organizational  performance  through  better  information  flow. 

Limitations  and  Future  Research 

This  project  only  considered  that  portion  of  the  AIMD  400  Division  that  accomplishes 
F414  maintenance.  It  considered  only  tasks  associated  with  maintenance  efforts  starting  from 
receipt  of  the  engine  to  the  point  at  which  the  engine  is  determined  ready  for  issue  (RFI). 
Although  other  maintenance  work  and  collateral  duties  not  directly  associated  with  F414 
maintenance  were  not  directly  modeled,  generic,  non-core  tasks  were  modeled  which  required 
personnel  to  perform  functions  other  than  F414  maintenance.  By  doing  so,  limitations  on  400 
Division  personnel’s  time  to  accomplish  F414  maintenance  were  accurately  characterized.  The 
scope  of  this  effort  was  further  limited  by  modeling  the  maintenance  of  only  a  single  engine, 
although  total  available  time  to  accomplish  tasks  were  correspondingly  decreased  to  that 
available  for  a  single  engine. 

Future  research  is  needed  to  track  AIMD  performance  post-implementation  of  selected 
interventions  and  to  compare  data  to  predicted  performance.  Other  organizations  within  the 
NAS  Lemoore  AIMD  (e.g.,  Airframe  Division,  Avionics  Division,  etc.)  should  also  be  separately 
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modeled  to  identify  potential  organizational  changes  which  may  improve  their  processes. 
Consideration  should  then  be  given  to  integrating  these  separate  models  to  develop  a  coherent 
AIMD  model  which  would  aid  in  identifying  modifications  to  the  larger  organization,  thus 
benefiting  information  flow.  The  model  developed  for  this  study  could  also  be  modified  to 
represent  engine  maintenance  divisions  in  other  AIMD  units  across  the  Navy. 

Conclusions 

This  study  in  applying  organizational  modeling  to  the  NAS  Lemoore  AIMD  identified 
several  potential  modifications  or  interventions  to  the  400  Division  which  could  reduce  F414 
engine-maintenance  throughput  time.  These  interventions  went  beyond  the  process- 
improvement  techniques  implemented  by  the  division  under  the  AIRSpeed  program  by  focusing 
primarily  on  improving  how  and  when  the  flow  of  information  through  the  organization  occurs. 

Results  have  shown  a  savings  between  16  and  46  days  of  maintenance  time  on  each 
engine:  an  average  of  26  days  per  engine.  The  Leadership  also  chose  to  partially  implement 
the  intervention  of  separately  combining  morning  and  afternoon  meetings.  Personnel  now 
receive  direction  on  required  daily  maintenance  actions  more  quickly,  which  has  increased  the 
amount  of  work  accomplished  each  day. 

Organizational  modeling  provided  key  insights  into  improving  the  NAS  Lemoore  AIMD 
F414  maintenance  process  and  allowed  management  to  consider  the  likely  impacts  of 
alternatives  on  time,  cost  and  quality  prior  to  making  these  changes.  The  significant 
improvement  in  reducing  F414  maintenance  throughput  time  that  resulted  from  this  study 
affords  high  confidence  in  achieving  future  improvements  in  other  Navy  maintenance 
organizations  via  the  tools  and  techniques  of  organizational  modeling. 

Organizational  modeling  has  great  potential  for  improving  on  outstanding  process- 
improvement  results  the  Navy  has  already  achieved  under  the  AIRSpeed  program. 
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